Python Hidden Gems

I am a fan of Raymond Hettinger. For those who do not know him, he is one of the Python core developers with significant contributions like itertools, generator expressions, sets manipulations, and others. But in my opinion, he is the kind of teacher who has a way of teaching you in an hour more than you would have ever imagined. I am a huge fan and always viewed his lectures and keynotes on youtube.

The latest one is from PyCon Italia 2022, where Raymond shares professional tips for writing great unit tests. I am sharing with you some of the notes that I kept while watching the video. How many of those have you used in your day-to-day programming?

Tips

1. If you can and have the option, use only py.test for testing. It is far superior to the standard library unittest module.

2. If you want to use a python linter use pyflakes.

3. Chains of andand orcan sometimes be expressed using any() and all():

a and b and d
all([a,b,d])
a or b or c or d
any([a,b,c,d])

4. Set Operators

• membership: element in someset
• union: combine 2 sets
• intersection: elements that are in both sets
• difference: sometimes called “complement”
• subset: everything in the first set is also in the second
• superset: everything in the second set is also in the first
• isdisjoint: datasets are non-overlapping

Except for sdisjoint(), all of these operators also apply to Counter() which is used for “multiset” operations

colors_needed = Counter([‘red’, ‘green’, ‘red’])
colors_available = Counter(red=3, blue=4, green=1)
colors_needed < colors_available

5. Reducers

We all know the usual ones (min(), max(), mean(), median(), len(),...) What caught my attention is mode() that I have never used it. The mode() is very good at finding dominant values. Think of it as a vote counter:

>>> mode([True, False, False, True, False, True, True, True, True])
True

6. Combinator Itertools

• Cartesian Product: generate all possible parameters for a function

filenames = [‘a.txt’, ‘b.txt’]
modes = [‘r’, ‘w’, ‘a’]
list(product(filenames, modes))
[('a.txt', 'r'),
('a.txt', 'w'),
('a.txt', 'a'),
('b.txt', 'r'),
('b.txt', 'w'),
('b.txt', 'a')]

• Permutations are useful for exploring all possible orderings of some user action:

actions = [1,2,3]
print(list(permutations(actions)))
[(1,2,3), 
 (1,3,2),
 (2,1,3),
 (2,3,1),
 (3,1,2),
 (3,2,1)]
# better example can be 
actions = ['press play', 'press stop', 'press eject', 'press rewind']

• Powerset() function is useful for generating all possible groupings:

users = ['adam', 'beth', 'cindy', 'dave']
pprint(list(powerset(users)))
[(),
 ('adam',),
 ('beth',),
 ('cindy',),
 ('dave', ),
 ('adam', 'beth'),
 ....
 ('adam', 'beth', 'cindy', 'dave')]

7. Capture

The following recipe is helpful when testing that 2 APIs have the same behavior including exceptions raised:

def capture(func, *args, **kwargs):
    try:
        return [None, func(*args, **kwargs)]
    except Exception as e:
        return [type(e), None]
for x in [-10, -1, 0, 0.2, 100, float('Inf'), float('Nan'), 'hi']:
    assert capture(log, x) == capture(altlog, x)

What do you feel about the above? Are you going to ever use it? The whole presentation is a gem and it also shows how to refactor code and make simpler and more concise test cases.