ken

python unpack objects

Python how to unpack tuple, list and dictionary

There are various cases that you want to unpack your python objects such as tuple, list or dictionary into individual variables, so that you can easily access the individual items. In this article I will be sharing with you how to unpack these different python objects and how it can be useful when working with the *args and **kwargs in the function.

Let’s get started.

Unpack python tuple objects

Let’s say we have a tuple object called shape which describes the height, width and channel of an image, we shall be able to unpack it to 3 separate variables by doing below:

shape = (500, 300, 3)
height, width, channel = shape
print(height, width, channel)

And you can see each item inside the tuple has been assigned to the individual variables with a meaningful name, which increases the readability of your code. Below is the output:

500 300 3

It’s definitely more elegant than accessing each items by index, e.g. shape[0], shape[1], shape[2].

What if we just need to access a few items in a big tuple which has many items? Here we need to introduce the _ (unnamed variable) and * (unpack arbitrary number of items)

For example,  if we just want to extract the first and the last item from the below tuple, we can let the rest of the items go into a unnamed variable.

toto_result = (4,11,14,23,28,47,24)
first, *_, last = toto_result
print(first, last)

So the above will give the below output:

4 24

If you are curious what is inside the “_”, you can try to print it out. and you would see it’s actually a list of the rest of items between the first and last item.

[11, 14, 23, 28, 47]

The most popular use case of the packing and unpacking is to pass around as parameters to function which accepts arbitrary number of arguments (*args). Let’s look at an example:

def sum(*numbers):
    total = 0
    for n in numbers:
        total += n
    return total

For the above sum function, it accepts any number of arguments and sum up the values. The * here is trying to pack all the arguments passed to this function and put it into a tuple called numbers. If you are going to sum up the values for all the items in toto_result, directly pass in the toto_result would not work.

toto_resut = (4,11,14,23,28,47,24)
#sum(toto_result) would raise TypeError

So what we can do is to unpack the items from the tuple then pass it the sum function:

total = sum(*toto_resut)
print(total)
#output should be 151

Unpack python list objects

Unpacking the list object is similar to the unpacking operations on tuple object. If we replace the tuple to list in the above example, it should be working perfectly.

shape = [500, 300, 3]
height, width, channel = shape
print(height, width, channel)
#output shall be 500 300 3

toto_result = [4,11,14,23,28,47,24]
first, *_, last = toto_result
print(first, last)
#output shall be 4 24

total = sum(*toto_resut) 
print(total) 
#output should be also 151

Unpack python dictionary objects

Unlike the list or tuple, unpacking the dictionary probably only useful when you wants to pass the dictionary as the keyword arguments into a function (**kwargs).

For instance, in the below function, you can pass in all your keyword arguments one by one.

def print_header(**headers):
    for header in headers:
        print(header, headers[header])

print_header(Host="Mozilla/5.0", referer = "https://www.codeforests.com")

Or if you have a dictionary like below, you can just unpack it and pass to the function:

headers = {'Host': 'www.codeforests.com', 'referer' : 'https://www.codeforests.com'}
print_header(**headers)

It will generate the same result as previously, but the code is more concise.

Host www.codeforests.com
referer https://www.codeforests.com

With this unpacking operator, you can also combine multiple dictionaries as per below:

headers = {'Host': 'www.codeforests.com', 'referer' : 'https://www.codeforests.com'}
extra_header = {'user-agent': 'Mozilla/5.0'}

new_header = {**headers, **extra_header}

The output of the new_header will be like below:

{'Host': 'www.codeforests.com',
 'referer': 'https://www.codeforests.com',
 'user-agent': 'Mozilla/5.0'}

Conclusion

The unpacking operation is very usefully especially when dealing with the *args and **kwargs. There is one thing worth noting on the unamed variable (_) which I mentioned in the previous paragraph. Please use it with caution, as if you notice, the python interactive interpreter also uses _ to store the last executed expression. So do take note on this potential conflict. See the below example:

codeforests interactive interpreter conflicts

As per always, welcome any comments or questions.

python read and write json file

Read and write json file in python

Json file format is commonly used in most of the programming languages to store data or exchange the data between back end and front end, or between different applications and systems. In this article, I will be explaining how to read and write json file in python programming language.

Read from a JSON file

Python has a json module which makes the read and write json pretty easy. First, let’s assume we have the below example.json file to be read.

{
"link": "www.codeforests.com",
"name": "ken", 
"member": true, 
"hobbies": ["jogging", "watching movie"]
}

To read the file, we can simply use the load method and pass in the file descriptor.

example = json.load(open("example.json"))

Now you can access the example dictionary for the data, e.g.

print(config["hobbies"])

The output would be :

['jogging', 'watching movie']

Write into JSON file

Let’s continue to use the previous example, and try to add one more hobby into the hobbies. Then save the json object into a file.

This time, you can use the json.dump and pass in the file descriptor to be written to:

example["hobbies"].append("badminton")
with open("example.json", "w") as f:
    json.dump(example, f)

If you look at the json documentation, there are two more methods : json.loads and json.dumps. The main difference of this two methods vs json.load & json.dumps is that the loads and dumps take the str representation of the json object. e.g.:

obj = json.loads('{"json":"obj"}')
print(obj)
print(json.dumps({"json":"obj"}))

 

How to read and write configuration (.ini) file in python

There are several file formats you can use for your configuration file, the most commonly used format are .ini, .json and .yaml. In this article, I will sharing with you how to read/write your configurations in the .ini file formats.

Read .ini file

Below is a example of the ini file, you can define the sections (e.g. [LOGIN]) as much as you want to separate the different configuration info.

[LOGIN]
user = admin
#Please change to your real password
password = admin

[SERVER]
host = 192.168.0.1
port = 8088

In python, there is already a module configparser to read an parse the information from the ini file int dictionary objects. Assume you have saved above as config.ini file into your current folder, you can use the below lines of code to read.

import configparser

config = configparser.ConfigParser()		
config.read("config.ini")
login = config['LOGIN']
server = config['SERVER']

You can assign each of the sections into a separate dictionary for easier accessing the values. The output should be same as below:

codeforests read ini file

Note that the line starting with # symbol (or ; ) will be taken as comment line and omitted when parsing the keys and values.

Also all the values are taken as string, so you will need to do your own data type conversion after you read it.

Write .ini file

Now let’s see how we can write to an ini file.

You will still need this configparser library, and the idea is that you need to set the keys and values into the configparser object and then save it into a file.

config = configparser.ConfigParser()
if not config.has_section("INFO"):
    config.add_section("INFO")
    config.set("INFO", "link", "www.codeforests.com")
    config.set("INFO", "name", "ken")

with open("example.ini", 'w') as configfile:
    config.write(configfile)

And this would create the example.ini file with below content:

[INFO]
link = www.codeforests.com
name = ken

I have created another two separate articles to cover the .json and .yaml format, please have a look if you are interested.

As per always, welcome any comments or questions.

Get file names by extension from a directory

Whenever you access the directories and files, you probably will need to implement some function to get file names by file extension from a particular directory. For instance, you may want to check and process all the excel files in a folder, or do a house keeping to remove all the old log files. In this article, I will be explaining to you a few ways of implementing such function.

Let’s get started!

There are actually plenty of libraries/modules you can use to achieve it, but let’s start with the most commonly used libraries/modules.

Option 1

Since you will need to import the os module anyway if you need to handle the file operations, you can make use of the functions from this module.

For instance, you can list out all the files/sub-directories under the current directory,  and check if file name ending with certain file extension as per below:

import os

pyfiles = []
for file in os.listdir("."):
    if file.lower().endswith(".ipynb"):
        pyfiles.append(file)

You can further sort the files by last modified time from latest to the earliest.

pyfiles.sort(key=os.path.getmtime, reverse=True)

What if you want to check multiple file extensions ? Don’t worries, you can still achieve it by some minor change on the if condition:

if file.lower().endswith((".ipynb", ".xlsx")):

Option 2

The os module also has another method scandir which is able to achieve the same, and also returns the file types and file attribute info.

files = []
for file in os.scandir("."):
    if file.name.lower().endswith((".ipynb", ".xlsx")):
        files.append(file.name)

 

Option 3

If you don’t like the way to match the file names in the above code, you can use fnmatch to do this job. for example: 

import fnmatch
files = []
for file in os.listdir("."):
    if fnmatch.fnmatch(file, "*.ipynb") or fnmatch.fnmatch(file, "*.xlsx"):
        files.append(file)

 

Option 4

Python has a glob module you can use the Unix style of pattern to match the files. To match the files with certain extension, you can simply do the below:

import glob
files = glob.glob("*.ipynb")

And then sort by the file creation from the latest to the earliest:

files.sort(key=os.path.getctime, reverse=True)

if you want match for multiple file extensions, you can do something as below:

files = []
file_types = ("*.ipynb", "*.xlsx")
for file_type in file_types:
    files.extend(glob.glob(file_type))

files.sort(key=os.path.getctime, reverse=True)

As I mentioned earlier, there are far more ways of doing it and it would not be possible to list of all them, so I will just stop here, and please leave your comments if you have better ideas.

 

How to swap key and value in a python dictionary

There are cases that you may want to swap key and value pair in a python dictionary, so that you can do some operation by using the unique values in the original dictionary.

For instance, if you have the below dictionary:

contact = {"joe" : "contact@company.com", "john": "john@company.com"}

you can swap key and value of the dictionary by:

contact = {val : key for key, val in contact.items()}
print(contact)

You will see the below output:

{'contact@company.com': 'joe', 'john@company.com': 'john'}

But for the above dictionary, if multiple names sharing the same email address, then only one name will be retained. e.g. :

contact = {"joe" : "contact@company.com", "jane" : "contact@company.com", "john": "john@company.com"}
contact = {val : key for key, val in contact.items()}

Output of the contact dictionary will be :

{'contact@company.com': 'jane', 'john@company.com': 'john'}

So how to keep all the keys that have the same value after reversing it ?

You will need to use a list or set to collect all the keys if the value is the same, e.g.:

email_contact = {}
for key, val in contact.items():
    email_contact.setdefault(val, []).append(key)

(please refer to this article about the setdefault method)

And you will see the below output for the new dictionary email_contact:

{'contact@company.com': ['joe', 'jane'], 'john@company.com': ['john']}

That’s exactly what we want ! Now we shall be able to say “hi” to both Joe and Jane when sending email to contact@company.com without missing any names.

 

As per always, welcome any comments or questions.