First, do not use mutable objects as function defaults
...: def_list.append(value)
...: return def_list
...:
In [2]: my_list = append_to_list(1)
In [3]: my_list
Out[3]: [1]
In [4]: my_other_list = append_to_list(2)
In [5]: my_other_list
Out[5]: [1, 2] # See, we were actually only going to generate [2] but instead we brought in the first run of the effects page
In [6]: import time
In [7]: def report_arg(my_default=()):
...: print(my_default)
...:
In [8]: report_arg() # first execution
1399562371.32
In [9]: (2) # 2 seconds apart
In [10]: report_arg()
1399562371.32 # It's amazing that time hasn't changed
What do these two examples show? Dictionaries, collections, lists, etc. are not suitable as default values for functions. This is because the default value is generated when the function is created, and each call uses the object's "cache". I've talked about this in my previous post on advanced python programming, but it's a real-world problem, so check the code you've been working on, it might just be a problem that hasn't been exposed yet!
You can change it like this.
def append_to_list(element, to=None):
if to is None:
to = []
(element)
return to
Second, the generator does not keep the results after iteration
In [13]: 2 in gen
Out[13]: True
In [14]: 3 in gen
Out[14]: True
In [15]: 1 in gen
Out[15]: False # Why is 1 not in gen anymore? Because 1->2 is called, and by this time 1 is no longer in the iterator, it's been generated on demand.
In [20]: gen = (i for i in range(5))
In [21]: a_list = list(gen) # can be converted to a list, but of course a_tuple = tuple(gen) would work too
In [22]: 2 in a_list
Out[22]: True
In [23]: 3 in a_list
Out[23]: True
In [24]: 1 in a_list # Even after loop, the value is still there.
Out[24]: True
Third, lambda saves local variables in closures
In [29]: my_list = [lambda: i for i in range(5)]
In [30]: for l in my_list:
....: print(l())
....:
4
4
4
4
4
The problem is the same as described above in advanced python programming. In fact, when I assign a value to my_list, the lambda expression executes a loop until i = 4, where i is retained.
But you can use the generator
In [31]: my_gen = (lambda: n for n in range(5))
In [32]: for l in my_gen:
....: print(l())
....:
0
1
2
3
4
You can also stick with LIST:
In [33]: my_list = [lambda x=i: x for i in range(5)] # See that I assigned default values to each lambda expression
In [34]: for l in my_list:
....: print(l())
....:
0
1
2
3
4
It's a little hard to understand, isn't it? Let's look at another piece of python magic.
In [35]: def groupby(items, size):
....: return zip(*[iter(items)]*size)
....:
In [36]: groupby(range(9), 3)
Out[36]: [(0, 1, 2), (3, 4, 5), (6, 7, 8)]
A grouping function. Seems like a lot to take in, doesn't it? Let's break it down here.
In [39]: [iter(items)]*3
Out[39]:
[<listiterator at 0x10e155fd0>,
<listiterator at 0x10e155fd0>,
<listiterator at 0x10e155fd0>] # See, in fact, it is the items into iterative, repeated three times (the same object Oh), but do not forget, each time.next(), so the role of grouping
In [40]: [lambda x=i: x for i in range(5)]
Out[40]:
[<function __main__.<lambda>>,
<function __main__.<lambda>>,
<function __main__.<lambda>>,
<function __main__.<lambda>>,
<function __main__. <lambda>>] # See?
IV. Modifying list items in a loop
In [45]: for i in a:
....: if not i % 2:
....: (i)
....:
In [46]: a
Out[46]: [1, 3, 5] # No questions asked
In [50]: b = [2, 4, 5, 6]
In [51]: for i in b:
....: if not i % 2:
....: (i)
....:
In [52]: b
Out[52]: [4, 5] # Originally I wanted the result to be a list with even numbers removed
Think about it, why - it's because you're removing the list, which affects its index.
In [53]: b = [2, 4, 5, 6]
In [54]: for index, item in enumerate(b):
....: print(index, item)
....: if not item % 2:
....: (item)
....:
(0, 2) # There's no problem here. 2 is deleted.
(1, 5) # Since 2 was deleted and the current list is [4, 5, 6], index list[1] directly to 5, ignoring 4.
(2, 6)
V. IndexError - list value exceeds his index number
In [55]: my_list = [1, 2, 3, 4, 5]
In [56]: my_list[5] # There is no such element.
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-56-037d00de8360> in <module>()
----> 1 my_list[5]
IndexError: list index out of range # It's throwing an exception.
In [57]: my_list[5:] # But it can be like this, be careful, it's a trick if you use it right, it's a pit if you use it wrong!
Out[57]: []
VI. Reusing global variables
In [58]: def my_func():
.... : print(var) # I can call an undefined variable first
....:
In [59]: var = 'global' # post-assignment
In [60]: my_func() # As long as the variable is defined when the function is called anyway
global
In [61]: def my_func():
....: var = 'locally changed'
....:
In [62]: var = 'global'
In [63]: my_func()
In [64]: print(var)
global # local variables do not affect global variables
In [65]: def my_func():
.... : print(var) # Even though you set the variable globally, there are local variables with the same name, and python thinks you forgot to define a local variable.
....: var = 'locally changed'
....:
In [66]: var = 'global'
In [67]: my_func()
---------------------------------------------------------------------------
UnboundLocalError Traceback (most recent call last)
<ipython-input-67-d82eda95de40> in <module>()
----> 1 my_func()
<ipython-input-65-0ad11d690936> in my_func()
1 def my_func():
----> 2 print(var)
3 var = 'locally changed'
4
UnboundLocalError: local variable 'var' referenced before assignment
In [68]: def my_func():
.... : global var # Gotta add the global at this point.
.... : print(var) # this will work fine
....: var = 'locally changed'
....:
In [69]: var = 'global'
In [70]:
In [70]: my_func()
global
In [71]: print(var)
locally changed # But using global changes the global variables.
VII. Copying mutable objects
In [73]: my_list1
Out[73]: [[1, 2, 3], [1, 2, 3]]
In [74]: my_list1[1][0] = 'a' # I'm only modifying one item in the sub list
In [75]: my_list1
Out[75]: [['a', 2, 3], ['a', 2, 3]] # But it all affects the
In [76]: my_list2 = [[1, 2, 3] for i in range(2)] # Using this loop to generate different objects won't hurt
In [77]: my_list2[1][0] = 'a'
In [78]: my_list2
Out[78]: [[1, 2, 3], ['a', 2, 3]]
VIII. python multiple inheritance (C3)
In [1]: class A(object):
...: def foo(self):
...: print("class A")
...:
In [2]: class B(object):
...: def foo(self):
...: print("class B")
...:
In [3]: class C(A, B):
...: pass
...:
In [4]: C().foo()
class A # Example is very understandable, C inherits A and B, from left to right, found that A has a foo method, returned the
It all looks very simple, sequentially searching from the bottom up, front to back, and returning when it's found. Here's another example.
In [5]: class A(object):
...: def foo(self):
...: print("class A")
...:
In [6]: class B(A):
...: pass
...:
In [7]: class C(A):
...: def foo(self):
...: print("class C")
...:
In [8]: class D(B,C):
...: pass
...:
In [9]: D().foo()
class C # ? By the way, the order is D->B->A, so why did you find out where C went?
This also brings us to MRO (Method Resolution Order).
In [10]: D.__mro__
Out[10]: (__main__.D, __main__.B, __main__.C, __main__.A, object)
A simple way to understand this is that the new class is now breadth-first, D->B, but when you realize that C also inherits from A, you look for C first, and then go to A at the end.
IX. list of + and + =, append and extend
('ID:', 4481323592)
In [18]: a_list += [1]
In [19]: print('ID (+=):', id(a_list))
('ID (+=):', 4481323592) # Using += still operates on the original list
In [20]: a_list = a_list + [2]
In [21]: print('ID (list = list + ...):', id(a_list))
('ID (list = list + ...) :', 4481293056) # The simple + has actually changed the original list
In [28]: a_list = []
In [29]: id(a_list)
Out[29]: 4481326976
In [30]: a_list.append(1)
In [31]: id(a_list)
Out[31]: 4481326976 # append is adding to the original list
In [32]: a_list.extend([2])
In [33]: id(a_list)
Out[33]: 4481326976 # extend also adds to the existing list
X. datetime also has a boolean value
It's a pit.
In [34]: import datetime
In [35]: print('"(0,0,0)" (Midnight) ->', bool((0,0,0)))
('"(0,0,0)" (Midnight) ->', False)
In [36]: print('"(1,0,0)" (1 am) ->', bool((1,0,0)))
('"(1,0,0)" (1 am) ->', True)
XI. The difference between '==' and is
My understanding is that "is" is to determine the identity of the 2 objects, == is to determine the value of the 2 objects.
In [37]: a = 1
In [38]: b = 1
In [39]: print('a is b', bool(a is b))
('a is b', True)
In [40]: c = 999
In [41]: d = 999
In [42]: print('c is d', bool(c is d))
('c is d', False) # The reason is python's memory management, which caches objects from -5 to 256.
In [43]: print('256 is 257-1', 256 is 257-1)
('256 is 257-1', True)
In [44]: print('257 is 258-1', 257 is 258 - 1)
('257 is 258-1', False)
In [45]: print('-5 is -6+1', -5 is -6+1)
('-5 is -6+1', True)
In [46]: print('-7 is -6-1', -7 is -6-1)
('-7 is -6-1', False)
In [47]: a = 'hello world!'
In [48]: b = 'hello world!'
In [49]: print('a is b,', a is b)
('a is b,', False) # Obviously they're not cached, it's a 2-field string object.
In [50]: print('a == b,', a == b)
('a == b,', True) # but they have the same value
# But, there's a special case
In [51]: a = float('nan')
In [52]: print('a is a,', a is a)
('a is a,', True)
In [53]: print('a == a,', a == a)
('a == a,', False) # Blinds me ~
XII. Shallow and deep copies
In practice, we can make changes to a list of objects, but we may not want to change the original list. Shallow copy only copies the parent object, while deep copy also copies the children inside the object.
In [65]: list1 = [1, 2]
In [66]: list2 = list1 # It's a reference, and if you manipulate list2, the result of list1 will also change.
In [67]: list3 = list1[:]
In [69]: import copy
In [70]: list4 = (list1) # He's a shallow copy like list3.
In [71]: id(list1), id(list2), id(list3), id(list4)
Out[71]: (4480620232, 4480620232, 4479667880, 4494894720)
In [72]: list2[0] = 3
In [73]: print('list1:', list1)
('list1:', [3, 2])
In [74]: list3[0] = 4
In [75]: list4[1] = 4
In [76]: print('list1:', list1)
('list1:', [3, 2]) # neither list3 nor list4 operations have an effect on list1
# And look at the difference between a deep copy and a shallow copy #
In [88]: from copy import copy, deepcopy
In [89]: list1 = [[1], [2]]
In [90]: list2 = copy(list1) # still a shallow copy
In [91]: list3 = deepcopy(list1) # deepcopy
In [92]: id(list1), id(list2), id(list3)
Out[92]: (4494896592, 4495349160, 4494896088)
In [93]: list2[0][0] = 3
In [94]: print('list1:', list1)
('list1:', [[3], [2]]) # See, if you manipulate its children, it's still the same as a reference, it affects the source.
In [95]: list3[0][0] = 5
In [96]: print('list1:', list1)
('list1:', [[3], [2]]) # Deep copy would have no effect on the
XIII. bool is actually a subclass of int
In [97]: isinstance(True, int)
Out[97]: True
In [98]: True + True
Out[98]: 2
In [99]: 3 * True + True
Out[99]: 4
In [100]: 3 * True - False
Out[100]: 3
In [104]: True << 10
Out[104]: 1024
XV. Is it true that tuples are immutable?
In [112]: tup[0] += [1]
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-112-d4f292cf35de> in <module>()
----> 1 tup[0] += [1]
TypeError: 'tuple' object does not support item assignment
In [113]: tup
Out[113]: ([1],) # Holy shit, you're blinding me again. You threw an exception and you're still able to modify it?
In [114]: tup = ([],)
In [115]: tup[0].extend([1])
In [116]: tup[0]
Out[116]: [1] # Well, I kinda see, I can't manipulate the tuple directly, but that doesn't stop me from manipulating the mutable subobjects of the tuple (the list).
Here's a nice explanation of Python's += Is Weird, Part II .
In [117]: my_tup = (1,)
In [118]: my_tup += (4,)
In [119]: my_tup = my_tup + (5,)
In [120]: my_tup
Out[120]: (1, 4, 5) # ? I thought you couldn't manipulate tuples?
In [121]: my_tup = (1,)
In [122]: print(id(my_tup))
4481317904
In [123]: my_tup += (4,)
In [124]: print(id(my_tup))
4480606864 # The operation is not on the original tuple so it can be
In [125]: my_tup = my_tup + (5,)
In [126]: print(id(my_tup))
4474234912
Python doesn't have private methods/variables? But there can be "pseudo" ones.
In [127]: class my_class(object^E):
.....: def public_method(self):
.....: print('Hello public world!')
..... : def __private_method(self): # private starts with double underscore
.....: print('Hello private world!')
.....: def call_private_method_in_class(self):
.....: self.__private_method()
In [132]: my_instance = my_class()
In [133]: my_instance.public_method()
Hello public world!
In [134]: my_instance._my_class__private_method()
Hello private world! # For private, add "_ + class name + private method name".
In [135]: my_instance.call_private_method_in_class()
Hello private world! # It can also be accessed internally through the public interface provided by the class.
In [136]: my_instance._my_class__private_variable
Out[136]: 1
XVII. Exception handling plus else
In [150]: try:
.....: print('third element:', a_list[2])
.....: except IndexError:
.....: print('raised IndexError')
.....: else:
..... : print('no error in try-block') # Execute the expression in else only if there is no exception in try
.....:
raised IndexError # Threw an exception. Not quite finished.
In [153]: i = 0
In [154]: while i < 2:
.....: print(i)
.....: i += 1
.....: else:
.....: print('in else')
.....:
0
1
in else # while is also supported
In [155]: i = 0
In [156]: while i < 2:
.....: print(i)
.....: i += 1
.....: break
.....: else:
.....: print('completed while-loop')
.....:
0 # broken, not fully executed, not executing the else.
In [158]: for i in range(2):
.....: print(i)
.....: else:
.....: print('completed for-loop')
.....:
0
1
completed for-loop
In [159]: for i in range(2):
.....: print(i)
.....: break
.....: else:
.....: print('completed for-loop')
.....:
0 # Also because of the break