1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
|
<span id="tut-brieftourtwo"></span><h1> Brief Tour of the Standard Library — Part II</h1> <p>This second tour covers more advanced modules that support professional programming needs. These modules rarely occur in small scripts.</p> <section id="output-formatting"> <span id="tut-output-formatting"></span><h2>
<span class="section-number">11.1. </span>Output Formatting</h2> <p>The <a class="reference internal" href="../library/reprlib#module-reprlib" title="reprlib: Alternate repr() implementation with size limits."><code>reprlib</code></a> module provides a version of <a class="reference internal" href="../library/functions#repr" title="repr"><code>repr()</code></a> customized for abbreviated displays of large or deeply nested containers:</p> <pre data-language="python">>>> import reprlib
>>> reprlib.repr(set('supercalifragilisticexpialidocious'))
"{'a', 'c', 'd', 'e', 'f', 'g', ...}"
</pre> <p>The <a class="reference internal" href="../library/pprint#module-pprint" title="pprint: Data pretty printer."><code>pprint</code></a> module offers more sophisticated control over printing both built-in and user defined objects in a way that is readable by the interpreter. When the result is longer than one line, the “pretty printer” adds line breaks and indentation to more clearly reveal data structure:</p> <pre data-language="python">>>> import pprint
>>> t = [[[['black', 'cyan'], 'white', ['green', 'red']], [['magenta',
... 'yellow'], 'blue']]]
...
>>> pprint.pprint(t, width=30)
[[[['black', 'cyan'],
'white',
['green', 'red']],
[['magenta', 'yellow'],
'blue']]]
</pre> <p>The <a class="reference internal" href="../library/textwrap#module-textwrap" title="textwrap: Text wrapping and filling"><code>textwrap</code></a> module formats paragraphs of text to fit a given screen width:</p> <pre data-language="python">>>> import textwrap
>>> doc = """The wrap() method is just like fill() except that it returns
... a list of strings instead of one big string with newlines to separate
... the wrapped lines."""
...
>>> print(textwrap.fill(doc, width=40))
The wrap() method is just like fill()
except that it returns a list of strings
instead of one big string with newlines
to separate the wrapped lines.
</pre> <p>The <a class="reference internal" href="../library/locale#module-locale" title="locale: Internationalization services."><code>locale</code></a> module accesses a database of culture specific data formats. The grouping attribute of locale’s format function provides a direct way of formatting numbers with group separators:</p> <pre data-language="python">>>> import locale
>>> locale.setlocale(locale.LC_ALL, 'English_United States.1252')
'English_United States.1252'
>>> conv = locale.localeconv() # get a mapping of conventions
>>> x = 1234567.8
>>> locale.format_string("%d", x, grouping=True)
'1,234,567'
>>> locale.format_string("%s%.*f", (conv['currency_symbol'],
... conv['frac_digits'], x), grouping=True)
'$1,234,567.80'
</pre> </section> <section id="templating"> <span id="tut-templating"></span><h2>
<span class="section-number">11.2. </span>Templating</h2> <p>The <a class="reference internal" href="../library/string#module-string" title="string: Common string operations."><code>string</code></a> module includes a versatile <a class="reference internal" href="../library/string#string.Template" title="string.Template"><code>Template</code></a> class with a simplified syntax suitable for editing by end-users. This allows users to customize their applications without having to alter the application.</p> <p>The format uses placeholder names formed by <code>$</code> with valid Python identifiers (alphanumeric characters and underscores). Surrounding the placeholder with braces allows it to be followed by more alphanumeric letters with no intervening spaces. Writing <code>$$</code> creates a single escaped <code>$</code>:</p> <pre data-language="python">>>> from string import Template
>>> t = Template('${village}folk send $$10 to $cause.')
>>> t.substitute(village='Nottingham', cause='the ditch fund')
'Nottinghamfolk send $10 to the ditch fund.'
</pre> <p>The <a class="reference internal" href="../library/string#string.Template.substitute" title="string.Template.substitute"><code>substitute()</code></a> method raises a <a class="reference internal" href="../library/exceptions#KeyError" title="KeyError"><code>KeyError</code></a> when a placeholder is not supplied in a dictionary or a keyword argument. For mail-merge style applications, user supplied data may be incomplete and the <a class="reference internal" href="../library/string#string.Template.safe_substitute" title="string.Template.safe_substitute"><code>safe_substitute()</code></a> method may be more appropriate — it will leave placeholders unchanged if data is missing:</p> <pre data-language="python">>>> t = Template('Return the $item to $owner.')
>>> d = dict(item='unladen swallow')
>>> t.substitute(d)
Traceback (most recent call last):
...
KeyError: 'owner'
>>> t.safe_substitute(d)
'Return the unladen swallow to $owner.'
</pre> <p>Template subclasses can specify a custom delimiter. For example, a batch renaming utility for a photo browser may elect to use percent signs for placeholders such as the current date, image sequence number, or file format:</p> <pre data-language="python">>>> import time, os.path
>>> photofiles = ['img_1074.jpg', 'img_1076.jpg', 'img_1077.jpg']
>>> class BatchRename(Template):
... delimiter = '%'
...
>>> fmt = input('Enter rename style (%d-date %n-seqnum %f-format): ')
Enter rename style (%d-date %n-seqnum %f-format): Ashley_%n%f
>>> t = BatchRename(fmt)
>>> date = time.strftime('%d%b%y')
>>> for i, filename in enumerate(photofiles):
... base, ext = os.path.splitext(filename)
... newname = t.substitute(d=date, n=i, f=ext)
... print('{0} --> {1}'.format(filename, newname))
img_1074.jpg --> Ashley_0.jpg
img_1076.jpg --> Ashley_1.jpg
img_1077.jpg --> Ashley_2.jpg
</pre> <p>Another application for templating is separating program logic from the details of multiple output formats. This makes it possible to substitute custom templates for XML files, plain text reports, and HTML web reports.</p> </section> <section id="working-with-binary-data-record-layouts"> <span id="tut-binary-formats"></span><h2>
<span class="section-number">11.3. </span>Working with Binary Data Record Layouts</h2> <p>The <a class="reference internal" href="../library/struct#module-struct" title="struct: Interpret bytes as packed binary data."><code>struct</code></a> module provides <a class="reference internal" href="../library/struct#struct.pack" title="struct.pack"><code>pack()</code></a> and <a class="reference internal" href="../library/struct#struct.unpack" title="struct.unpack"><code>unpack()</code></a> functions for working with variable length binary record formats. The following example shows how to loop through header information in a ZIP file without using the <a class="reference internal" href="../library/zipfile#module-zipfile" title="zipfile: Read and write ZIP-format archive files."><code>zipfile</code></a> module. Pack codes <code>"H"</code> and <code>"I"</code> represent two and four byte unsigned numbers respectively. The <code>"<"</code> indicates that they are standard size and in little-endian byte order:</p> <pre data-language="python">import struct
with open('myfile.zip', 'rb') as f:
data = f.read()
start = 0
for i in range(3): # show the first 3 file headers
start += 14
fields = struct.unpack('<IIIHH', data[start:start+16])
crc32, comp_size, uncomp_size, filenamesize, extra_size = fields
start += 16
filename = data[start:start+filenamesize]
start += filenamesize
extra = data[start:start+extra_size]
print(filename, hex(crc32), comp_size, uncomp_size)
start += extra_size + comp_size # skip to the next header
</pre> </section> <section id="multi-threading"> <span id="tut-multi-threading"></span><h2>
<span class="section-number">11.4. </span>Multi-threading</h2> <p>Threading is a technique for decoupling tasks which are not sequentially dependent. Threads can be used to improve the responsiveness of applications that accept user input while other tasks run in the background. A related use case is running I/O in parallel with computations in another thread.</p> <p>The following code shows how the high level <a class="reference internal" href="../library/threading#module-threading" title="threading: Thread-based parallelism."><code>threading</code></a> module can run tasks in background while the main program continues to run:</p> <pre data-language="python">import threading, zipfile
class AsyncZip(threading.Thread):
def __init__(self, infile, outfile):
threading.Thread.__init__(self)
self.infile = infile
self.outfile = outfile
def run(self):
f = zipfile.ZipFile(self.outfile, 'w', zipfile.ZIP_DEFLATED)
f.write(self.infile)
f.close()
print('Finished background zip of:', self.infile)
background = AsyncZip('mydata.txt', 'myarchive.zip')
background.start()
print('The main program continues to run in foreground.')
background.join() # Wait for the background task to finish
print('Main program waited until background was done.')
</pre> <p>The principal challenge of multi-threaded applications is coordinating threads that share data or other resources. To that end, the threading module provides a number of synchronization primitives including locks, events, condition variables, and semaphores.</p> <p>While those tools are powerful, minor design errors can result in problems that are difficult to reproduce. So, the preferred approach to task coordination is to concentrate all access to a resource in a single thread and then use the <a class="reference internal" href="../library/queue#module-queue" title="queue: A synchronized queue class."><code>queue</code></a> module to feed that thread with requests from other threads. Applications using <a class="reference internal" href="../library/queue#queue.Queue" title="queue.Queue"><code>Queue</code></a> objects for inter-thread communication and coordination are easier to design, more readable, and more reliable.</p> </section> <section id="logging"> <span id="tut-logging"></span><h2>
<span class="section-number">11.5. </span>Logging</h2> <p>The <a class="reference internal" href="../library/logging#module-logging" title="logging: Flexible event logging system for applications."><code>logging</code></a> module offers a full featured and flexible logging system. At its simplest, log messages are sent to a file or to <code>sys.stderr</code>:</p> <pre data-language="python">import logging
logging.debug('Debugging information')
logging.info('Informational message')
logging.warning('Warning:config file %s not found', 'server.conf')
logging.error('Error occurred')
logging.critical('Critical error -- shutting down')
</pre> <p>This produces the following output:</p> <pre data-language="none">WARNING:root:Warning:config file server.conf not found
ERROR:root:Error occurred
CRITICAL:root:Critical error -- shutting down
</pre> <p>By default, informational and debugging messages are suppressed and the output is sent to standard error. Other output options include routing messages through email, datagrams, sockets, or to an HTTP Server. New filters can select different routing based on message priority: <a class="reference internal" href="../library/logging#logging.DEBUG" title="logging.DEBUG"><code>DEBUG</code></a>, <a class="reference internal" href="../library/logging#logging.INFO" title="logging.INFO"><code>INFO</code></a>, <a class="reference internal" href="../library/logging#logging.WARNING" title="logging.WARNING"><code>WARNING</code></a>, <a class="reference internal" href="../library/logging#logging.ERROR" title="logging.ERROR"><code>ERROR</code></a>, and <a class="reference internal" href="../library/logging#logging.CRITICAL" title="logging.CRITICAL"><code>CRITICAL</code></a>.</p> <p>The logging system can be configured directly from Python or can be loaded from a user editable configuration file for customized logging without altering the application.</p> </section> <section id="weak-references"> <span id="tut-weak-references"></span><h2>
<span class="section-number">11.6. </span>Weak References</h2> <p>Python does automatic memory management (reference counting for most objects and <a class="reference internal" href="../glossary#term-garbage-collection"><span class="xref std std-term">garbage collection</span></a> to eliminate cycles). The memory is freed shortly after the last reference to it has been eliminated.</p> <p>This approach works fine for most applications but occasionally there is a need to track objects only as long as they are being used by something else. Unfortunately, just tracking them creates a reference that makes them permanent. The <a class="reference internal" href="../library/weakref#module-weakref" title="weakref: Support for weak references and weak dictionaries."><code>weakref</code></a> module provides tools for tracking objects without creating a reference. When the object is no longer needed, it is automatically removed from a weakref table and a callback is triggered for weakref objects. Typical applications include caching objects that are expensive to create:</p> <pre data-language="python">>>> import weakref, gc
>>> class A:
... def __init__(self, value):
... self.value = value
... def __repr__(self):
... return str(self.value)
...
>>> a = A(10) # create a reference
>>> d = weakref.WeakValueDictionary()
>>> d['primary'] = a # does not create a reference
>>> d['primary'] # fetch the object if it is still alive
10
>>> del a # remove the one reference
>>> gc.collect() # run garbage collection right away
0
>>> d['primary'] # entry was automatically removed
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
d['primary'] # entry was automatically removed
File "C:/python312/lib/weakref.py", line 46, in __getitem__
o = self.data[key]()
KeyError: 'primary'
</pre> </section> <section id="tools-for-working-with-lists"> <span id="tut-list-tools"></span><h2>
<span class="section-number">11.7. </span>Tools for Working with Lists</h2> <p>Many data structure needs can be met with the built-in list type. However, sometimes there is a need for alternative implementations with different performance trade-offs.</p> <p>The <a class="reference internal" href="../library/array#module-array" title="array: Space efficient arrays of uniformly typed numeric values."><code>array</code></a> module provides an <a class="reference internal" href="../library/array#array.array" title="array.array"><code>array()</code></a> object that is like a list that stores only homogeneous data and stores it more compactly. The following example shows an array of numbers stored as two byte unsigned binary numbers (typecode <code>"H"</code>) rather than the usual 16 bytes per entry for regular lists of Python int objects:</p> <pre data-language="python">>>> from array import array
>>> a = array('H', [4000, 10, 700, 22222])
>>> sum(a)
26932
>>> a[1:3]
array('H', [10, 700])
</pre> <p>The <a class="reference internal" href="../library/collections#module-collections" title="collections: Container datatypes"><code>collections</code></a> module provides a <a class="reference internal" href="../library/collections#collections.deque" title="collections.deque"><code>deque()</code></a> object that is like a list with faster appends and pops from the left side but slower lookups in the middle. These objects are well suited for implementing queues and breadth first tree searches:</p> <pre data-language="python">>>> from collections import deque
>>> d = deque(["task1", "task2", "task3"])
>>> d.append("task4")
>>> print("Handling", d.popleft())
Handling task1
</pre> <pre data-language="python">unsearched = deque([starting_node])
def breadth_first_search(unsearched):
node = unsearched.popleft()
for m in gen_moves(node):
if is_goal(m):
return m
unsearched.append(m)
</pre> <p>In addition to alternative list implementations, the library also offers other tools such as the <a class="reference internal" href="../library/bisect#module-bisect" title="bisect: Array bisection algorithms for binary searching."><code>bisect</code></a> module with functions for manipulating sorted lists:</p> <pre data-language="python">>>> import bisect
>>> scores = [(100, 'perl'), (200, 'tcl'), (400, 'lua'), (500, 'python')]
>>> bisect.insort(scores, (300, 'ruby'))
>>> scores
[(100, 'perl'), (200, 'tcl'), (300, 'ruby'), (400, 'lua'), (500, 'python')]
</pre> <p>The <a class="reference internal" href="../library/heapq#module-heapq" title="heapq: Heap queue algorithm (a.k.a. priority queue)."><code>heapq</code></a> module provides functions for implementing heaps based on regular lists. The lowest valued entry is always kept at position zero. This is useful for applications which repeatedly access the smallest element but do not want to run a full list sort:</p> <pre data-language="python">>>> from heapq import heapify, heappop, heappush
>>> data = [1, 3, 5, 7, 9, 2, 4, 6, 8, 0]
>>> heapify(data) # rearrange the list into heap order
>>> heappush(data, -5) # add a new entry
>>> [heappop(data) for i in range(3)] # fetch the three smallest entries
[-5, 0, 1]
</pre> </section> <section id="decimal-floating-point-arithmetic"> <span id="tut-decimal-fp"></span><h2>
<span class="section-number">11.8. </span>Decimal Floating Point Arithmetic</h2> <p>The <a class="reference internal" href="../library/decimal#module-decimal" title="decimal: Implementation of the General Decimal Arithmetic Specification."><code>decimal</code></a> module offers a <a class="reference internal" href="../library/decimal#decimal.Decimal" title="decimal.Decimal"><code>Decimal</code></a> datatype for decimal floating point arithmetic. Compared to the built-in <a class="reference internal" href="../library/functions#float" title="float"><code>float</code></a> implementation of binary floating point, the class is especially helpful for</p> <ul class="simple"> <li>financial applications and other uses which require exact decimal representation,</li> <li>control over precision,</li> <li>control over rounding to meet legal or regulatory requirements,</li> <li>tracking of significant decimal places, or</li> <li>applications where the user expects the results to match calculations done by hand.</li> </ul> <p>For example, calculating a 5% tax on a 70 cent phone charge gives different results in decimal floating point and binary floating point. The difference becomes significant if the results are rounded to the nearest cent:</p> <pre data-language="python">>>> from decimal import *
>>> round(Decimal('0.70') * Decimal('1.05'), 2)
Decimal('0.74')
>>> round(.70 * 1.05, 2)
0.73
</pre> <p>The <a class="reference internal" href="../library/decimal#decimal.Decimal" title="decimal.Decimal"><code>Decimal</code></a> result keeps a trailing zero, automatically inferring four place significance from multiplicands with two place significance. Decimal reproduces mathematics as done by hand and avoids issues that can arise when binary floating point cannot exactly represent decimal quantities.</p> <p>Exact representation enables the <a class="reference internal" href="../library/decimal#decimal.Decimal" title="decimal.Decimal"><code>Decimal</code></a> class to perform modulo calculations and equality tests that are unsuitable for binary floating point:</p> <pre data-language="python">>>> Decimal('1.00') % Decimal('.10')
Decimal('0.00')
>>> 1.00 % 0.10
0.09999999999999995
>>> sum([Decimal('0.1')]*10) == Decimal('1.0')
True
>>> 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 == 1.0
False
</pre> <p>The <a class="reference internal" href="../library/decimal#module-decimal" title="decimal: Implementation of the General Decimal Arithmetic Specification."><code>decimal</code></a> module provides arithmetic with as much precision as needed:</p> <pre data-language="python">>>> getcontext().prec = 36
>>> Decimal(1) / Decimal(7)
Decimal('0.142857142857142857142857142857142857')
</pre> </section> <div class="_attribution">
<p class="_attribution-p">
© 2001–2023 Python Software Foundation<br>Licensed under the PSF License.<br>
<a href="https://docs.python.org/3.12/tutorial/stdlib2.html" class="_attribution-link">https://docs.python.org/3.12/tutorial/stdlib2.html</a>
</p>
</div>
|
