Callables in Python

Introduction

A callable is an object in Python that can be called / executed when called with parantheses ( ). Classes and functions are callable.

Callables can be a class, a function, or an instance of a class. In simple terms, a class/function/instance/builtin is callable if it gets executed while being called with the parantheses ().

Example 1:


In [1]: help()
Welcome to Python 3.6's help utility!
-- content omitted --
--------
In [2]: int()
Out[2]: 0

In [3]: callable(int)
Out[3]: True

--------
In [4]: callable(help)
Out[4]: True

--------
In [5]: def hello():
        ..: print("Howdy!!")

In [6]: hello()
Howdy!!

In [7]: callable(hello)
Out[7]: True

In Example 1, we can see that the builtins like help(), a pre-defined type such as int(), and a custom function hello() are all callable. These can be executed while being called with parantheses.

The __call__() method

The callable() builtin helps to determine if an object is callable or not. Internally, it translates to the magic method __call__().

In short:

my_object(*args) translates to my_object.__call__(*args)

All classes and functions are callable, as well as instances of classes with the __call__ magic method. An instance of a class/function is usually not callable (even though the class/function itself is), unless the class carries a __call__ magic method.

In other words, an instance is callable only if the class it is instantiated from contains the __call__ magic method.

  • The inbuilt documentation on callable states:
In [1]: print(callable.__doc__)
Return whether the object is callable (i.e., some kind of function).

Note that classes are callable, as are instances of classes with a
__call__() method.

Example 2:


In [5]: def hello():
       ...: print("Howdy!!")

In [6]: hello()
Howdy!!

In [7]: hello.__call__()
Howdy!!

In [8]: callable(hello)
Out[8]: True

Example 2 shows that a function when called with the parantheses (including any required arguments) is equivalent to calling the __call__() magic method. ie.. calling a function/class with parantheses translates to calling the __call__() magic method.

NOTE:
Read more on Magic methods in Python

Example 3: Non-callable Instances


In [1]: type(1)
Out[1]: int

In [2]: callable(1)
Out[2]: False

In [3]: x = 1

In [4]: type(x)
Out[4]: int

In [5]: callable(int)
Out[5]: True

In [6]: callable(x)
Out[6]: False

Example 3 above shows that even though the int class is callable, the instances created from the int class are not.

Remember that instances will only be callable if the class from which it was instantiated contains a __call__ method. Inspecting the methods of class int reveals that it does not have a __call__ method.

NOTE: You can view the methods of the int class using help(int) or dir(int).

Example 4: Another example with Classes


In [52]: class tell:
        ...: def __call__(self):
            ...: pass

In [53]: telling = tell()

In [54]: callable(tell)
Out[54]: True

In [55]: callable(telling)
Out[55]: True

--------

In [56]: class test:
        ...: pass

In [57]: testing = test()

In [58]: callable(test)
Out[58]: True

In [59]: callable(testing)
Out[59]: False

Since all classes are by default callable, both the classes tell and test in Example 4 are callable. But the instances of these classes necessarily need not be so. Since the class tell has the magic method __call__, the instance telling is callable. But the instance testing instantiated from the class test is not since the class does not have the magic method. Another set of examples.

Example 5: Non-callable instance of a class


In [1]: class new:
       ...: def foo(self):
           ...: print("Hello")

In [2]: n = new()

In [3]: n()
------------------
TypeError Traceback (most recent call last)
 in module()
----> 1 n()

TypeError: 'new' object is not callable


Example 6: Callable instance of the same class

In [4]: class new:
...: def __call__(self):
    ...: print("I'm callable!")

In [5]: n = new()

In [6]: n
Out[6]: __main__.new at 0x7f7a614b1f98

In [7]: n()
I'm callable!

Example 5  and Example 6 shows how a class is itself callable, but unless it carries a __call__() method, the instances spawned out of it are not so.

 

References:

  1. http://docs.python.org/3/library/functions.html#callable
  2. http://eli.thegreenplace.net/2012/03/23/python-internals-how-callables-work/
  3. https://docs.python.org/3/reference/datamodel.html#object.__call__

Recursion – Algorithm Study

Recursion is a technique by which a function calls itself until a condition is met.

Introduction

Loops or repetitive execution based on certain conditions are inevitable in programs. Usual loops include if, while and for loops. Recursion is an entirely different way to deal with such situations, and in many cases, easier.

Recursion is a when a function calls itself in each iteration till a condition is met. Ideally, the data set in each iteration gets smaller until it reach the required condition, after which the recursive function exists.

A typical example of recursion is a factorial function.

How does Recursion work?

A recursive function ideally contains a Base case and a Recursive case.

Recursive case is when the function calls itself, until the Base case is met. Each level of iteration in the Recursive case moves the control to the next level.

Once a specific level finishes execution, the control is passed back to the previous level of execution. A Recursive function can go several layers deep until the Base condition is met. In short, a Recursive case is a loop in which the function calls itself.

The Base case is required so that the function doesn’t continue running in the Recursive loop forever. Once the Base case is met, the control moves out of the Recursive case, executes the conditions in the Base case (if any), and exits.

As mentioned in the Introduction, a factorial function can be seen as an example of recursion.

NOTE:

The Base case for a factorial function is when n == 1

Consider n!:

n! can be written as:

n x (n – 1) x (n – 2) x (n – 3) x …. x 1

n! can also be represented as:

 n!       = n * (n - 1)! ---> [Step 1]
(n - 1)! = (n - 1) * (n - 2)! ---> [Step 2]
(n - 2)! = (n - 2) * (n - 3)! ---> [Step 3]
.
..
...
(n - (n - 1)) = 1 ---> [Base case]

Each level/step is a product of a value and all the levels below it. Hence, Step 1 will end up moving to Step 2 to get the factorial of elements below it, then to Step 3 and so on.

ie.. the control of execution move as:

[Step 1] -> [Step 2] -> [Step 3] -> ….. [Step n]

In a much easier-to-grasp example, a 5! would be:

5! = 5 * 4! ---> [Step 1]
4! = 4 * 3! ---> [Step 2]
3! = 3 * 2! ---> [Step 3]
2! = 2 * 1! ---> [Step 4]
1! = 1      ---> [Step 5] / [Base case]

The order of execution will be :

[Step 1] -> [Step 2] -> [Step 3] -> [Step 4] -> [Step 5]

As we know, in Recursion, each layer pause itself and pass the control to the next level. Once it reach the end or the Base case, it returns the result back to the previous level one by one until it reaches where it started off.

In this example, once the control of execution reaches Step 5 / Base case ,  the control is returned back to its previous level Step 4 . This level returns the result back to Step 3 which completes its execution and returns to Step 2 , so on and so forth until it reach  Step 1 .

The return control flow would be as:

[Base case / Step 5] -> [Step 4] -> [Step 3] -> [Step 2] -> [Step 1] -> Result.

This can be summed up using an awesome pictorial representation, from the book Grokking Algorithms by Adit. Please check out the References section for the link for more information about this awesome book.

factorial_recursion

Figure 1: Recursion, Recursive case and Base case (Copyright Manning Publications, drawn by adit.io)

 Code

Example 1:

  • A factorial function in a while loop
def fact(n):
factorial = 1
while n > 1:
factorial = factorial * n
n = n - 1
return factorial

print("Factorial of {0} is {1}".format(10, fact(10)))
print("Factorial of {0} is {1}".format(20, fact(20)))
  • The same function above, in a recursive loop
def factorial(n):
    if n == 0:
        return 1
    else:
        return n * factorial(n - 1)

print("Factorial of {0} is {1}".format(10, factorial(10)))
print("Factorial of {0} is {1}".format(20, factorial(20)))

Example 2:

  • A function to sum numbers in a normal for loop.
def my_sum(my_list):
    num = 0
    for i in my_list:
        num += i
    return num

print(my_sum([10, 23, 14, 12, 11, 94, 20]))
  • The same function to add numbers, in a recursive loop
def my_sum(my_list):
    if my_list == []:
        return 0
    else:
        return my_list[0] + my_sum(my_list[1:])

print(my_sum([10, 23, 14, 12, 11, 94, 20]))

Code explanation

Both Example 1 and Example 2 are represented as an iterative function as well as a recursive function.

The iterative function calls the next() function on the iterator sum.__iter__() magic method iterate over the entire data set. The recursive function calls itself to reach a base case and return the result.

Observations:

While a recursive function does not necessarily give you an edge on performance, it is much easier to understand and the code is cleaner.

Recursion has a disadvantage though, for large data sets. Each loop is put on a call stack until it reaches a Base case. Once the Base case is met, the call stack is rewound back to reach where it started, executing each of the previous levels on the way. The examples above showed a sum function and a factorial function. In large data sets, this can lead to a large call stack which in turns take a lot of memory.

References:

  1. Grokking Algorithms
  2. Data Structures and Algorithms in Python

 

Selection Sort – Algorithm Study

Selection Sort is a sorting algorithm used to sort a data set either in incremental or decremental order.

It goes through the entire elements one by one and hence it’s not a very efficient algorithm to work on large data sets.

How does Selection sort work?

Selection sort starts with an unsorted data set. With each iteration, it builds up a sub dataset with the sorted data.

By the end of the sorting process, the sub dataset contains the entire elements in a sorted order.

1. Iterate through the data set one element at a time.
2. Find the biggest element in the data set (Append it to another if needed)
3. Reduce the sample space by the biggest element just found. The new data set becomes `n – 1` compared to the previous iteration.
4. Start over the iteration again, on the reduced sample space.
5. Continue till we have a sorted data set, either incremental or decremental

How does the data sample change in each iteration?

Consider the data set [10, 4, 9, 3, 6, 19, 8]

[10, 4, 9, 3, 6, 19, 8]      – Data set
[10, 4, 9, 3, 6, 8] – [19] – After Iteration 1
[4, 9, 3, 6, 8] – [10, 19] – After Iteration 2
[4, 3, 6, 8] – [9, 10, 19] – After Iteration 3
[4, 3, 6] – [8, 9, 10, 19] – After Iteration 4
[4, 3] – [6, 8, 9, 10, 19] – After Iteration 5
[3] – [4, 6, 8, 9, 10, 19] – After Iteration 6
[3, 4, 6, 8, 9, 10, 19]      – After Iteration 7 – Sorted data set

Let’s check what the Selection Sort algorithm has to go through in each iteration.

Dataset – [10, 4, 9, 3, 6, 19, 8]
Iter 1 – [10, 4, 9, 3, 6, 8]
Iter 2 – [4, 9, 3, 6, 8]
Iter 3 – [4, 3, 6, 8]
Iter 4 – [4, 3, 6]
Iter 5 – [4, 3]
Iter 6 – [3]

Sorted Dataset – [3, 4, 6, 8, 9, 10, 19]

Performance / Time Complexity

Selection Sort has to go through all the elements in the data set, no matter what.

Hence, the Worst case, Best case and Average Time Complexity would be O(n^2).

Since `Selection Sort` takes in `n` elements while starting, and goes through the data set `n` times (each step reducing the data set size by 1 member), the iterations would be:

n + [ (n – 1) + (n – 2) + (n – 3) + (n – 4) + … + 2 + 1 ]

We are more interested in the worse-case scenario. In a very large data set, an `n – 1`, `n – 2` etc.. won’t make a difference.

Hence we can re-write the above iterations as:

n + [n + n + n + n ….. n]

Or also as:

n * n = (n^2)

Code

def find_smallest(my_list):
    smallest = my_list[0]
    smallest_index = 0

for i in range(1, len(my_list)):
    if my_list[i] < smallest:
        smallest = my_list[i]
        smallest_index = i
return smallest_index

def selection_sort(my_list):
    new_list = []
    for i in range(len(my_list)):
        smallest = find_smallest(my_list)
        new_list.append(my_list.pop(smallest))
return new_list

Observations:

1.Selection Sort is an algorithm to sort a data set, but it is not particularly fast.
2. It takes `n` iterations in each step to find the biggest element in that iteration.
3. The next iteration has to run on a data set of `n – 1` elements compared to the previous iteration.
4. For `n` elements in a sample space, Selection Sort takes `n x n` iterations to sort the data set.

References:

1. https://en.wikipedia.org/wiki/Selection_sort
2. http://bigocheatsheet.com
3. https://github.com/egonschiele/grokking_algorithms

 

Accessor and Mutator methods – Python

A method defined within a class can either be an Accessor or a Mutator method.

An Accessor method returns the information about the object, but do not change the state or the object.

A Mutator method, also called an Update method, can change the state of the object.

Consider the following example:

In [10]: a = [1,2,3,4,5]

In [11]: a.count(1)
Out[11]: 1

In [12]: a.index(2)
Out[12]: 1

In [13]: a
Out[13]: [1, 2, 3, 4, 5]

In [14]: a.append(6)

In [15]: a
Out[15]: [1, 2, 3, 4, 5, 6]

The methods a.count() and a.index() are both Accessor methods since it doesn’t alter the object a in any sense, but only pulls the relevant information.

But a.append() is a mutator method, since it effectively changes the object (list a) to a new one.

In short, knowing the behavior of a method is helpful to understand how it alters the objects it acts upon.

Inheritance and super() – Object Oriented Programming

super() is a feature through which inherited methods can be accessed, which has been overridden in a class. It can also help with the MRO lookup order in case of multiple inheritance. This may not be obvious first, but a few examples should help to drive the point home.

Inheritance and method overloading was discussed in a previous post, where we saw how inherited methods can be overloaded or enhanced in the child classes.

In many scenarios, it’s needed to overload an inherited method, but also call the actual method defined in the Parent class.

Let’s start off with a simple example based on Inheritance, and build from there.

Example 0:

class MyClass(object):

    def func(self):
        print("I'm being called from the Parent class!")

class ChildClass(MyClass):
    pass

my_instance_1 = ChildClass()
my_instance_1.func()

This outputs:

In [18]: %run /tmp/super-1.py
I'm being called from the Parent class

In Example 0, we have two classes, MyClass and ChildClass. The latter inherits from the former, and the parent class MyClass has a method named func defined.

Since ChildClass inherits from MyClass, the child class has access to the methods defined in the parent class. An instance is created my_instance_2, for ChildClass.

Calling my_instance_1.func() will print the statement from the Parent class, due to the inheritance.

Building up on the first example:

Example 1:

class MyClass(object):

    def func(self):
        print("I'm being called from the Parent class")

class ChildClass(MyClass):

    def func(self):
        print("I'm being called from the Child class")

my_instance_1 = MyClass()
my_instance_2 = ChildClass()

my_instance_1.func()
my_instance_2.func()

This outputs:

In [19]: %run /tmp/super-1.py
I'm being called from the Parent class
I'm being called from the Child class

This example has a slight difference, both the child class as well as the parent class have the same method defined, ie.. func. In this scenario, the parent class’ method is overridden by the child class method.

ie.. if we call the func() method from the instance of ChildClass, it need not go a fetch the method from its Parent class, since it’s already defined locally.

NOTE: This is due to the Method Resolution Order, discussed in an earlier post.

But what if there is a scenario that warranties the need for specifically calling methods defined in the Parent class, from the instance of a child class?

ie.. How to call the methods defined in the Parent class, through the instance of the Child class, even if the Parent class method is overloaded in the Child class?

In such a case, the inbuilt function super() can be used. Let’s add to the previous example.

Example 2:

class MyClass(object):

    def func(self):
        print("I'm being called from the Parent class")

class ChildClass(MyClass):

    def func(self):
        print("I'm actually being called from the Child class")
        print("But...")
        # Calling the `func()` method from the Parent class.
        super(ChildClass, self).func()

my_instance_2 = ChildClass()
my_instance_2.func()

This outputs:

In [21]: %run /tmp/super-1.py
I'm actually being called from the Child class
But...
I'm being called from the Parent class

How is the code structured?

  1. We have two classes MyClass and ChildClass.
  2. The latter is inheriting from the former.
  3. Both classes have a method named func
  4. The child class ChildClass is instantiated as my_instance_2
  5. The func method is called from the instance.

How does the code work?

  1. When the func method is called, the interpreter searches it using the Method Resolution Order, and find the method defined in the class ChildClass.
  2. Since it finds the method in the child class, it executes it, and prints the string “I’m actually being called from the Child class”, as well “But…”
  3. The next statement is super which calls the method func defined in the parent class of ChildClass
  4. Since the control is now passed onto the func method in the Parent class via super, the corresponding print() statement is printed to stdout.

Example 2 can also be re-written as :

class MyClass(object):

    def func(self):
        print("I'm being called from the Parent class")

class ChildClass(MyClass):

    def func(self):
        print("I'm actually being called from the Child class")
        print("But...")
        # Calling the `func()` method from the Parent class.
        # super(ChildClass, self).func()
        MyClass.func(self)  # Call the method directly via Parent class

my_instance_2 = ChildClass()
my_instance_2.func()

 

NOTE: The example above uses the Parent class directly to access it’s method. Even though it works, it is not the best way to do it since the code is tied to the Parent class name. If the Parent class name changes, the child/sub class code has to be changed as well. 

Let’s see another example for  super() . This is from our previous article on Inheritance and method overloading.

Example 3:

import abc

class MyClass(object):

    __metaclass__ = abc.ABCMeta

    def my_set_val(self, value):
        self.value = value

    def my_get_val(self):
        return self.value

    @abc.abstractmethod
    def print_doc(self):
        return

class MyChildClass(MyClass):

    def my_set_val(self, value):
        if not isinstance(value, int):
            value = 0
        super(MyChildClass, self).my_set_val(self)

    def print_doc(self):
        print("Documentation for MyChild Class")

my_instance = MyChildClass()
my_instance.my_set_val(100)
print(my_instance.my_get_val())
print(my_instance.print_doc())

The code is already discussed here. The my_set_val method is defined in both the child class as well as the parent class.

We overload the my_set_val method defined in the parent class, in the child class. But after enhancing/overloading it, we call the my_set_val method specifically from the Parent class using super() and thus enhance it.

Takeaway:

  1. super() helps to specifically call the Parent class method which has been overridden in the child class, from the child class.
  2. The super() in-built function can be used to call/refer the Parent class without explicitly naming them. This helps in situations where the Parent class name may change. Hence, super() helps in avoiding strong ties with class names and increases maintainability.
  3. super() helps the most when there are multiple inheritance happening, and the MRO ends up being complex. In case you need to call a method from a specific parent class, use super().
  4. There are multiple ways to call a method from a Parent class.
    1. <Parent-Class>.<method>
    2. super(<ChildClass>, self).<method>
    3. super().<method>

References:

  1. https://docs.python.org/2/library/functions.html#super
  2. https://rhettinger.wordpress.com/2011/05/26/super-considered-super/
  3. https://stackoverflow.com/questions/222877/how-to-use-super-in-python

Abstract Base Classes/Methods – Object Oriented Programming

Abstract classes, in short, are classes that are supposed to be inherited or subclassed, rather than instantiated.

Through Abstract Classes, we can enforce a blueprint on the subclasses that inherit the Abstract Class. This means that Abstract classes can be used to define a set of methods that must be implemented by it subclasses.

Abstract classes are used when working on large projects where classes have to be inherited, and need to strictly follow certain blueprints.

Python supports Abstract Classes via the module abc from version 2.6. Using the abc module, its pretty straight forward to implement an Abstract Class.

Example 0:

import abc

class My_ABC_Class(object):
    __metaclass__ = abc.ABCMeta

    @abc.abstractmethod
    def set_val(self, val):
        return

    @abc.abstractmethod
    def get_val(self):
        return

# Abstract Base Class defined above ^^^

# Custom class inheriting from the above Abstract Base Class, below

class MyClass(My_ABC_Class):

    def set_val(self, input):
        self.val = input

    def get_val(self):
        print("\nCalling the get_val() method")
        print("I'm part of the Abstract Methods defined in My_ABC_Class()")
        return self.val

    def hello(self):
        print("\nCalling the hello() method")
        print("I'm *not* part of the Abstract Methods defined in My_ABC_Class()")

my_class = MyClass()

my_class.set_val(10)
print(my_class.get_val())
my_class.hello()

In the code above, set_val() and get_val() are both abstract methods defined in the Abstract Class My_ABC_Class(). Hence it should be implemented in the child class inheriting from My_ABC_Class().

In the child class MyClass() , we have to strictly define the abstract classes defined in the Parent class. But the child class is free to implement other methods of their own. The hello() method is one such.

This will print :

# python abstractclasses-1.py

Calling the get_val() method
I'm part of the Abstract Methods defined in My_ABC_Class()
10

Calling the hello() method
I'm *not* part of the Abstract Methods defined in My_ABC_Class()

The code gets executed properly even if the  hello() method is not an abstract method.

Let’s check what happens if we don’t implement a method marked as an abstract method, in the child class.

Example 1:

import abc

class My_ABC_Class(object):
    __metaclass__ = abc.ABCMeta

    @abc.abstractmethod
    def set_val(self, val):
        return

    @abc.abstractmethod
    def get_val(self):
        return

# Abstract Base Class defined above ^^^

# Custom class inheriting from the above Abstract Base Class, below

class MyClass(My_ABC_Class):

    def set_val(self, input):
        self.val = input

    def hello(self):
        print("\nCalling the hello() method")
        print("I'm *not* part of the Abstract Methods defined in My_ABC_Class()")

my_class = MyClass()

my_class.set_val(10)
print(my_class.get_val())
my_class.hello()

Example 1 is the same as Example 0 except we don’t have the get_val() method defined in the child class.

This means that we’re breaking the rule of abstraction. Let’s see what happens:

# python abstractclasses-2.py
Traceback (most recent call last):
  File "abstractclasses-2.py", line 50, in
    my_class = MyClass()
TypeError: Can't instantiate abstract class MyClass with abstract methods get_val

The traceback clearly states that the child class MyClass() cannot be instantiated since it does not implement the Abstract methods defined in it’s Parent class.

We mentioned that an Abstract class is supposed to be inherited rather than instantiated. What happens if we try instantiating an Abstract class?

Let’s use the same example, this time we’re instantiating the Abstract class though.

Example 2:

import abc

class My_ABC_Class(object):
    __metaclass__ = abc.ABCMeta

    @abc.abstractmethod
    def set_val(self, val):
        return

    @abc.abstractmethod
        def get_val(self):
            return

# Abstract Base Class defined above ^^^

# Custom class inheriting from the above Abstract Base Class, below

class MyClass(My_ABC_Class):

    def set_val(self, input):
        self.val = input

    def hello(self):
        print("\nCalling the hello() method")
        print("I'm *not* part of the Abstract Methods defined in My_ABC_Class()")

my_class = My_ABC_Class()    # <- Instantiating the Abstract Class

my_class.set_val(10)
print(my_class.get_val())
my_class.hello()

What does this output?

# python abstractclasses-3.py
Traceback (most recent call last):
    File "abstractclasses-3.py", line 54, in <module>
       my_class = My_ABC_Class()
TypeError: Can't instantiate abstract class My_ABC_Class with abstract methods get_val, set_val

As expected, the Python interpreter says that it can’t instantiate the abstract class My_ABC_Class.

Takeaway: 

  1. An Abstract Class is supposed to be inherited, not instantiated.
  2. The Abstraction nomenclature is applied on the methods within a Class.
  3. The abstraction is enforced on methods which are marked with the decorator @abstractmethod or @abc.abstractmethod, depending on how you imported the module, from abc import abstractmethod or import abc.
  4. It is not mandatory to have all the methods defined as abstract methods, in an Abstract Class.
  5. Subclasses/Child classes are enforced to define the methods which are marked with @abstractmethod in the Parent class.
  6. Subclasses are free to create methods of their own, other than the abstract methods enforced by the Parent class.

Reference:

  1. https://pymotw.com/2/abc/
  2. Python beyond the basics – Object Oriented Programming

Magic methods and Syntactic sugar in Python

Magic methods

Magic methods are special methods which can be defined (or already designed and available) to act on objects.

Magic methods start and end with underscores "__", and are not implicitly called by the user even though they can be. Most magic methods are used as syntactic sugar by binding it to more clear/easy_to_understand keywords.

Python is mostly objects and method calls done on objects. Many available functions in Python are actually tied to magic methodsLet’s checkout a few examples.

Example 0:

In [1]: my_var = "Hello!"

In [2]: print(my_var)
Hello!

In [3]: my_var.__repr__()
Out[3]: "'Hello!'"

As we can see, the __repr__() magic method can be called to print the object, ie.. it is bound to the print() keyword.

This is true for many other builtin keywords/operators as well.

Example 1:

In [22]: my_var = "Hello, "
In [23]: my_var1 = "How are you?"

In [24]: my_var + my_var1
Out[24]: 'Hello, How are you?'

In [25]: my_var.__add__(my_var1)
Out[25]: 'Hello, How are you?'

Here, Python interprets the + sign as a mapping to the magic method __add__(), and calls it on the L-value (Left hand object value) my_var, with the R-value (Right hand object value) as the argument.

When a builtin function is called on an object, in many cases it is mapped to the magic method.

Example 2:

In [69]: my_list_1 = ['a', 'b', 'c', 'd']

In [70]: 'a' in my_list_1
Out[70]: True

In [71]: my_list_1.__contains__("a")
Out[71]: True

The in builtin is mapped to the __contains__()method.

The methods available for an object should mostly be dependent on the type of the object.

Example 3:

In [59]: my_num = 1

In [60]: type(my_num)
Out[60]: int

In [61]: my_num.__doc__
Out[61]: Out[61]: "int(x=0) -> int or long\nint(x, base=10) -> int or long\n\nConvert a number or string to an integer, or return 0 if no arguments\nare given. ....>>>

In [62]: help(my_num)
class int(object)
| int(x=0) -> int or long
| int(x, base=10) -> int or long
|
| Convert a number or string to an integer, or return 0 if no arguments
| are given. If x is floating point, the conversion truncates towards zero.
| If x is outside the integer range, the function returns a long instead.

From the tests above, we can understand that the help() function is actually mapped to the object.__doc__ magic method. It’s the same doc string that __doc__ and help() uses.

NOTE: Due to the syntax conversion (+ to __add__(),and other conversions), operators like + , in, etc.. are also called Syntactic sugar.

What is Syntactic sugar?

According to Wikipedia, Syntact sugar is:

In computer science, syntactic sugar is syntax within a programming language that is designed to make things easier to read or to express. It makes the language “sweeter” for human use: things can be expressed more clearly, more concisely, or in an alternative style that some may prefer.

Hence, magic methods can be said to be Syntactic sugar. But it’s not just magic methods that are mapped to syntactic sugar methods, but higher order features such as Decorators are as well.

Example 4: 

def my_decorator(my_function):
    def inner_decorator():
        print("This happened before!")
        my_function()
        print("This happens after ")
        print("This happened at the end!")
    return inner_decorator

def my_decorated():
    print("This happened!")

var = my_decorator(my_decorated)

if __name__ == '__main__':
    var()

The example above borrows from one of the examples in the post on Decorators.

Here, my_decorator() is a decorator and is used to decorate my_decorated(). But rather than calling the decorator function my_decorator() with the argument my_decorated(), the above code can be syntactically sugar-coated as below:

def my_decorator(my_function):
    def inner_decorator():
        print("This happened before!")
        my_function()
        print("This happens after ")
        print("This happened at the end!")
    return inner_decorator

@my_decorator
def my_decorated():
    print("This happened!")

if __name__ == '__main__':
    my_decorated()

Observing both code snippets, the decorator is syntactically sugar coated and called as:

@my_decorator

instead of instantiating the decorator with the function to be decorated as an argument, ie..

var = my_decorator(my_decorated)

A few syntax resolution methods:

  1. ‘name’ in my_list       ->      my_list.__contains__(‘name’)
  2. len(my_list)                  ->      my_list.__len__()
  3. print(my_list)              ->      my_list.__repr__()
  4. my_list == “value”     ->      my_list.__eq__(“value”)
  5. my_list[5]                      ->      my_list.__getitem__(5)
  6. my_list[5:10]                 ->     my_list.__getslice__(5, 10)

NOTE: This article is written from the notes created while learning magic methods. The following articles (along with several others) were referred as part of the process.

  1. A Guide to Python’s Magic Methods, by Rafe Kettler
  2. Special method names, The Official Python 3 documentation