Python

Python interact with smart contract, python brownie, python mint NFT, python call smart contract function

Python Interact with Smart Contract

In the previous post, we have discussed how to deploy a NFT smart contract with Python Brownie. Once your contract is deployed into the blockchain, you can interact with your live contract either by building up your own Dapp or using scripts to perform some repetitive tasks like batch token transferring or airdrop. In this article, we will share with you some popular approaches to interact with smart contract in Python once the contract is live in public network.

Prerequisite

If you do not have Brownie installed, you shall go ahead to run the below pip command in your own Python virtual environment to install it:

#run in your venv
pip install eth-brownie

Brownie is heavily relying on Web3.py and use ganache cli as the default emulator for the development network, so you will get these dependencies installed automatically.

And assuming you have already compiled your smart contract and run the deployment script without any issue, if not, you may check through this post before proceeding. Now let’s explore how to interact with a deployed contract.

Work with Contract in Ganache Local Network

Using Ganache CLI

The Ganache CLI is used by default for development, and the local network is spin up when you are running your deployment script and terminated as soon as the script finished execution. In this case, the deployed contract is not stored anywhere permanently and you can only interact with it within same session before Ganache is shut down. The deployed contract is added into Brownie’s deployment map automatically, so you do not have to note down contract address when you want to refer to it. For instance, you can get your last deployed contract as per below:

# assuming a smart contract LegendNFT has been deployed
last_deployed_contract = LegendNFT[-1]

I would recommend you to use Brownie console when you just get started. It’s a Python interactive mode with all Brownie stuff loaded, and you can immediately get feedbacks if something goes wrong in your code. When everything is tested ok in the console, you can then copy your code to the Python script and make it re-usable for later use.

To start Brownie console for your local network, you can use the below command:

brownie console

And from the console, you can also deploy your contract:

>>> contract = LegendNFT.deploy({'from' : accounts[0]})

Transaction sent: 0x38643bde3b513dbaedf887c60162e5bb915b32bde2a804172dfaa2520b4d4ac8
Gas price: 0.0 gwei Gas limit: 12000000 Nonce: 0
LegendNFT.constructor confirmed Block: 1 Gas used: 3188048 (26.57%)
LegendNFT deployed at: 0x3194cBDC3dbcd3E11a07892e7bA5c3394048Cc87

Then you can verify that the contract address returned from deploy function is the same as the last record stored in Brownie’s deployment map:

>>> LegendNFT[-1]
<LegendNFT Contract '0x3194cBDC3dbcd3E11a07892e7bA5c3394048Cc87'>
>>> contract == LegendNFT[-1]
True

In this case, you can always use LegendNFT[-1] to get the last deployed contract instead of replying on the contract address returned from your deploy function, and the deployment script can be kept separately so that you don’t accidently redeploying the contract when you want to run your other functions.

One advantage of using console is that you can use dir function to inspect what are the available functions in the contract instance, e.g.:

>>> dir(LegendNFT[-1])
[abi, address, approve, balance, balanceOf, bytecode, decode_input, getApproved, 
getTotalNFTsMintedSoFar, get_method, get_method_object, info, isApprovedForAll, 
makeLegendNFT, name, ownerOf, safeTransferFrom, selectors, setApprovalForAll, 
signatures, supportsInterface, symbol, tokenURI, topics, totalSupply, transferFrom, tx]

Some of these functions are not explicated implemented in the contract but inherited from other contracts, you will be able to call them as long as the functions have public or external visibility specifier.

Now you can call whatever functions available in your contract, for instance:

>>> contract.name()
'Codeforests Legend'
>>> contract.totalSupply()
1000000

>>> contract.getTotalNFTsMintedSoFar()
0

And you can also call your mint function which will change the state of the blockchain:

>>> contract.makeLegendNFT({'from':accounts[0]})

Transaction sent: 0x6dabe95e4c54d43864e91bfeb398c2bfb0549bf48d51db2cd42b9882e501a905
  Gas price: 0.0 gwei   Gas limit: 12000000   Nonce: 1
  LegendNFT.makeLegendNFT confirmed   Block: 2   Gas used: 777946 (6.48%)

<Transaction '0x6dabe95e4c54d43864e91bfeb398c2bfb0549bf48d51db2cd42b9882e501a905'>

Of course you would still not be able to see how it looks like for your newly minted NFT since the minting was only happening in your local network. If you use the tokenURI function and pass in the first token ID, you shall see some base64 encoded data similar to below:

>>> contract.tokenURI(0)

'data:application/json;base64,eyJuYW1lIjogIlNwbGVuZGlkIEMjIFByb2dyYW0iLCAiZGVzY3JpcHRpb24iOiAiQSBsZWdlbmQgTkZUIGNvbGxlY3Rpb24gZm9yIGFsbCBjb2RlZm9yZXN0cyByZWFkZXJzLiIsICJpbWFnZSI6ICJkYXRhOmltYWdlL3N2Zyt4bWw7YmFzZTY0LFBITjJaeUI0Yld4dWN6MG5hSFIwY0RvdkwzZDNkeTUzTXk1dmNtY3ZNakF3TUM5emRtY25JSEJ5WlhObGNuWmxRWE53WldOMFVtRjBhVzg5SjNoTmFXNVpUV2x1SUcxbFpYUW5JSFpwWlhkQ2IzZzlKekFnTUNBek5UQWdNelV3Sno0OGMzUjViR1UrTG1KaGMyVWdleUJtYVd4c09pQWpSa1ZHUlVaRk95Qm1iMjUwTFdaaGJXbHNlVG9nVEdWaFozVmxJRk53WVhKMFlXNDdJR1p2Ym5RdGMybDZaVG9nTWpSd2VEc2dmVHd2YzNSNWJHVStQSEpsWTNRZ2QybGtkR2c5SnpFd01DVW5JR2hsYVdkb2REMG5NVEF3SlNjZ1ptbHNiRDBuSXpBMU1UWXlNaWNnTHo0OGRHVjRkQ0I0UFNjMU1DVW5JSGs5SnpVd0pTY2dZMnhoYzNNOUoySmhjMlVuSUdSdmJXbHVZVzUwTFdKaGMyVnNhVzVsUFNkdGFXUmtiR1VuSUhSbGVIUXRZVzVqYUc5eVBTZHRhV1JrYkdVblBsTndiR1Z1Wkdsa0lFTWpJRkJ5YjJkeVlXMDhMM1JsZUhRK1BDOXpkbWMrIn0='

This is the URI info for your NFT, and you can use base64 decoder to decode the data to see more details, or use this website to preview the NFT content.

Using Ganache UI

If you are using Ganache desktop version, you can add a new network in Brownie and let Brownie to connect to your Ganache RPC client. Below is the command to add a new network:

brownie networks add Development ganache-local host=http://127.0.0.1 network_id=5777 port=7545 cmd=ganache-cli

The network ID/host/port can be found from your Ganache client as per below:

Python Interact with Smart Contract, Python mint NFT, Python Brownie, Ganache client

Once you have added the network, you can start your Brownie console as per below:

brownie console --network ganache-local

#Attached to local RPC client listening at '127.0.0.1:7545'...

And then deploy your contract:

>>> LegendNFT.deploy({'from' : accounts[0]})

You shall see some transactions went through if you check the Ganache client:

Python Interact with Smart Contract, Python mint NFT, Python Brownie, Ganache client

So you will be able to trace the transaction history from Ganache client, and once you restart it, those history data will be gone and you will get a fresh new blockchain environment same as how the Ganache CLI works.

Work with Smart Contract in Public Network

When you use Brownie to deploy your contracts to public network such as rinkeby or mainnet, it automatically organize the contract ABI file by chain ID into the build/deployments folder as per below:

Python Interact with Smart Contract, Python mint NFT, Python Brownie, Ganache client

So as long as you do not delete these files, you can use ContractName[-1] to get the last deployed contract. To connect to rinkeby in Brownie console, specify the network as rinkeby:

brownie console --network rinkeby

Then you can interact with your contract same as working in the local network except you will need to load a real wallet. If you want to be sure you are working on the correct contract, you can use LegendNFT.at with deployed contract address instead of LegendNFT[-1] :

Python Interact with Smart Contract, Python mint NFT, Python Brownie, Ganache client

Awesome! you can now mint your NFT from your deployed contract in a public network. Probably you don’t want to always type the same code in the console window when you want to mint a new NFT. so let’s put the code we have tested into a script called mint.py in our scripts folder as per below:

from brownie import network, config, accounts, LegendNFT

def get_account():
    if network.show_active() in ["development", "ganache-local"]:
        return accounts[0]
    else:
        return accounts.add(config["wallets"]["from_key"])

def mint_nft():
    contract = LegendNFT[-1]
    account = get_account()
    tx = contract.makeLegendNFT({"from" : account})
    tx.wait(1)
    print("Minted a new NFT with txn hash:", tx.txid)

def main():
    mint_nft()

So when we are running this script in local network, it will just pick a dummy account for deployment, but when working in public network, it will load the account from our config for deployment.

Let’s run this script with below:

brownie run mint.py --network rinkeby

You shall see similar output as we did from console window:

Python Interact with Smart Contract, Python mint NFT, Python Brownie, Ganache client

And here is the transaction details we can see from the etherscan by searching the transaction hash.

What if I have accidentally deleted my ABI file or I want to access the contract created by someone else where I do not have the ABI data?

Don’t worried, as long as you have your contract address, you will still be able to access the contract. You can use the below from_explorer from the Contract object to fetch the information from network explorer. For instance:

>>> contract = Contract.from_explorer("0xaa629be88190fe2077b970aa02f52da40ce98454")
Fetching source of 0xaA629BE88190fe2077b970aa02f52dA40cE98454 from api-rinkeby.etherscan.io...

>>> contract.name()
'Codeforests Legend'

And if you want to try the hard way, you can actually copy the ABI data from the network explorer such as etherscan, ployscan etc. into a local file, then load it manually into a JSON object. For instance, I have saved the ABI data from here into a file called abi_from_rinkeby.json:

Python Interact with Smart Contract, Python mint NFT, Python Brownie, Smart Contract ABI

and with Brownie console connected to Rinkeby,  I can access my contract as per below:

>>> import json
>>> with open("./scripts/abi_from_rinkeby.json") as file:
...     contract_abi = json.load(file)
...
>>> contract = Contract.from_abi("LegendNFT", "0xaA629BE88190fe2077b970aa02f52dA40cE98454", contract_abi)
>>>
>>> contract.totalSupply()
1000000
>>>
>>> contract.getTotalNFTsMintedSoFar()
3

Now with the above approach, you can start mint an NFT from someone else’s contract without going through their website.

Conclusion

In this article, we have reviewed through a few approaches you can use to interact with your smart contract. Brownie is built up on top of the Web3.py, but you can still use the APIs from the web3 module to interact with your contract. There are also quite a few of popular JavaScript libraries such as web3.js, ethers.js etc., you may take a look if you are planning to interact with the contract from your frontend JavaScript code.

brownie deploy smart contract python, NFT smart contract, mint NFT, free NFT

Deploy Your First NFT with Python

Introduction

NFT (Non-Fungible Token) has exploded since last year and keeps on roaring regardless of how the cryptocurrency is performing. You must have heard of CryptoPunk, BAYC or the most recent Phantom Bear if you are following any crypto news. In the nutshell, NFT is a smart contract with some meta data such as image, video or some other attribute data to make it unique from one another. And since it’s on blockchain, it’s easy for anybody to verify the ownership. In this article, I will walk you through the process on how to deploy your own NFT with Python and we will be using a Python based smart contract development framework called Brownie.

Prerequisite

You will need to install the Brownie package. It is recommended to use pipx to install it in an isolated environment, but since I have my own virtual environment created, I would continue to use pip instead. Below is the pip command to install Brownie:

#run in your venv
pip install eth-brownie

In order to test your code locally, Brownie will automatically install ganache cli which launches a local Ethereum blockchain for you to execute your smart contracts. You can follow its documentation to download and install it if you wish to install it manually.

Get Started with Brownie

Let’s create an empty folder and then run the below command to initiate a new Brownie project:

brownie init

With the above command, brownie will create a project structure for you under your current folder. The majority of your work shall be done in the following folders:

  • contracts – for solidity smart contract source code
  • interfaces – for the interface files that referenced by the smart contract
  • scripts – for deployment scripts or interacting with deployed contracts
  • tests – for test scripts

 

Brownie supports both Solidity and Vyper language for creating smart contracts. In this article, I will be using the smart contract in Solidity style as the example.

Create NFT Smart Contract

NFTs use ERC721 token standard, so we will need to create a smart contract that implements this standard. Since the objective of this article is to walk through the process rather than a deep dive into the smart contract coding, I have just shared a sample smart contract here.  You can download this LegendNFT.sol smart contract file and the libraries folder, then put them under the contracts folder.

Here is a quick explanation of what this smart contract does:

  • an ERC721 contract with name Codeforests Legend, token symbol CFL and total supply of 1,000,000
  • 3 lists of words to generate a “random” word phrase
  • a SVG xml template to create a picture of the above word phrase
  • a list of color codes to provide “random” background color to the SVG
  • a mint function for minting the NFT tokens
  • finally a function to return how many tokens have minted

 

As you can see from the source code that I used two external contracts from OpenZeppelin, so I will need to specify this dependency so that they can be recognized when compiling the contract. Brownie provides a config file brownie-config.yaml which allows you specify these configurations. This file is not created automatically when initiating the project as all the settings are optional, you will only need it when necessary.

To specify the dependencies, you can manually create this yaml file under your project root folder. And add the below configurations:

dependencies:
  - OpenZeppelin/[email protected]
compiler:
  solc:
    remappings:
      - '@openzeppelin=OpenZeppelin/[email protected]'

It basically tells Brownie package manager to automatically download this package for you and let compiler find the correct path of the package when using the short form ‘import @openzeppelin/…’.

Compile Smart Contract

Once you have saved your contract and the config file, you can run below compile command on your terminal:

brownie compile

You shall see brownie auto downloaded the dependencies and compiled the code:

brownie compile smart contract, NFT smart contract

The output Json files will be placed into the build/contracts folder. The next step we shall try to deploy this contract locally.

Deploy Smart Contract Locally

To deploy our contract locally, we will need to use the ganache as I mentioned in the beginning. By default, when you want to deploy a contract, brownie will always start the ganache local network and deploy it to the local network. Brownie provides a console window where you can use the commands to deploy contract, but since we may need to do the deployment multiples for testing or in different networks, so let’s create a re-usable Python script called deploy.py under the scripts folder for the deployment purpose.

When your smart contract is compiled, the contract class object will be automatically added to brownie runtime environment, so we can import it from brownie directly.

Below is the Python code for deploying my LegendNFT contract:

from brownie import LegendNFT, network, config, accounts


def deploy_contract():
    account = accounts[0]
    legend_contract = LegendNFT.deploy({"from" : account})
    print("contract has been deployed successfully to :", legend_contract.address)

    return legend_contract

def main():
    deploy_contract()

The accounts is some dummy accounts provided by ganache only for your local network, so we just pick the first dummy account as the contract creator for the from parameter.

Note that your script needs to have a main function to let brownie know which function to run.

Next, let’s run the below command in the terminal:

brownie run .\scripts\deploy.py

You shall see something similar to the below output showing that both contract address and transaction hash have been generated:

brownie deploy smart contract python, NFT smart contract

Congratulations that you’ve just deployed your contract to your local network, although you may have no idea if the contract works correctly. Don’t worry, let’s continue to explore how to deploy to a public network and come back to the unit test later.

Deploy Smart Contract to Public Network

The public network usually refers to the public accessible and persistent blockchain network such as the Ethereum mainnet, rinkeby, kovan, popsten etc. To access the public network, you will need to connect to a node, but it would be too much to run your own node just for deploying a contract.

Luckily there are some service providers like Infura, Alchemy or Pocket Network etc., who allows you to use their API to connect to the public network nodes. For Brownie, it uses Infura as the default provider for connecting to public network. If you run below command in your terminal:

brownie networks list true

You can see it is using Infura’s API:

brownie deploy smart contract python, NFT smart contract, rinkeby network

In this case, you will need to sign up Infura for a free account and get your own project API token. Once you have your API token, you can create a environment variable called WEB3_INFURA_PROJECT_ID and assign your token ID to it. My personal preference would be putting this information into a .env file and load it into the Brownie config file.

And another thing you need is a real wallet account with some fund, as deployment will always incur some cost for confirming the transaction. You can create a wallet with Metamask, and for testnet like rinkeby, you can get some testing ether token from chainlink rinkeby faucet. You can also find other faucets for the rest of the testnets.

So let’s put below info in the .env file:

PRIVATE_KEY = 0x{your wallet private key}
WEB3_INFURA_PROJECT_ID = {your infrua project ID}
ETHERSCAN_TOKEN = {etherscan API token; to submit your contract source code to etherscan (optional)}

Note: to add 0x before your wallet private key so that it’s recognized as hex string. If you wish to get your contract source code verified by etherscan, you will need to sign up with etherscan and get an API token as well.

And add below lines in the brownie-config.yaml file:

dotenv: .env

wallets:
  from_key: ${PRIVATE_KEY}

# optional for verifying the contract source code
networks:
  development:
    verify : false
  rinkeby:
    verify : true

I strongly recommend you to add .env in .gitignore, so that you won’t accidently push your sensitive information into github. Your wallet private key for testnet and mainnet is the same, so bear in mind to not expose to anyone.

Now let’s also do some minor change to our deployment script, so that it uses the new account we created for deploying the contract to public network.

def deploy_contract():
    if(network.show_active() == "development"):
        account = accounts[0]
    else:
        account = accounts.add(config["wallets"]["from_key"])

    legend_contract = LegendNFT.deploy(
        {"from" : account}, 
        publish_source=config["networks"][network.show_active()].get("verify")
    )
    print("contract has been deployed successfully to :", legend_contract.address)

    return legend_contract

We have added a condition to check whether we are working on local/development network, when it’s not local network, we will load our account by the private key and use it to sign the transaction for deployment.

Now we have everything ready, let’s run the deploy.py again and specify the target network with the –network option, e.g. :

brownie run .\scripts\deploy.py --network rinkeby

Below is the output from my terminal:

brownie deploy smart contract python, NFT smart contract, Python deploy smart contract to public network

It would take slightly longer time since it requires the transaction to be minted and confirmed. You can copy the contract address or transaction hash then search it from etherscan rinkeby network.

You can see my deployed contract address here and the verified source code here.

Awesome! You just had your contract deployed in the public network where everybody can see it!!! Since you’ve not yet minted any NFT from your contract, you won’t see anything visually at the moment. To give you a idea of how your NFTs will look like, I have created this website for minting the LegendNFT, you can try and experience the minting process and then view the NFT from Opeasea. It currently supports Rinkeby, Matic and Matic Mumbai network. Below is the minting page after you have connected your wallet:

brownie deploy smart contract python, NFT smart contract, mint NFT, free NFT

Conclusion

Creating your own NFT can be a very fun journey although you may not be able to eventually sell it on the market. And there are just too much details to be covered and which is impossible to write in one post, so I will try to write a few more articles on how to perform unit test, interact with your deployed contract, as well as deploy the contract to other EVM compatible networks. Do follow me on my twitter and let me know if you encounter any issue with your contract deployment.

pandas convert columns to rows, convert wide to long, pandas melt

Pandas Tips – Convert Columns To Rows

  Introduction

In one of my previous posts – Pandas tricks to split one row of data into multiple rows, we have discussed a solution to split the summary data from one row into multiple rows in order to standardize the data for further analysis. Similarly, there are many scenarios that we have the aggregated data like a Excel pivot table, and we need to unpivot it from wide to long format for better analysis. In this article, I will be sharing with you a few tips to convert columns to rows with pandas DataFrame.

Prerequisites

To run the later code examples, you shall get pandas installed in your working environment. Below is the pip command to install pandas:

pip install pandas

And we will be using the data from this file for the later demonstration, so you may download and examine how the data looks like with below code:

import pandas as pd
import os
data_dir = "c:\\your_download_dir"
df = pd.read_excel(os.path.join(data_dir, "Sample-Data.xlsx"))

You shall see the sample sales data as per below:

pandas convert columns to rows, wide to long format, pandas melt

The sales amount has been summarized by each product in the last 4 columns. With this wide data format, it would be difficult for us to do some analysis, for instance, the top salesman by month by products or the best seller products by month etc.

A better data format should be transforming the product columns into rows so that each single row only represents 1 product and its sales amount. Now let’s start to explore what are the different ways to convert columns to rows with pandas.

Using Pandas Stack Method

The most immediate solution you may think of would be using the stack method as it allows you to stack the columns vertically onto each other and make it into multiple rows.  For our case, we will need to specify the DataFrame index as “Salesman” and “Order Date“, so that the product columns will stack based on this index. For instance:

df.set_index(["Salesman", "Order Date"]).stack()

If you check the result now, you shall see the below output:

pandas convert columns to rows, wide to long format, pandas melt

This is an MultiIndex Series with index name – [‘Salesman’, ‘Order Date’, None], so you can reset the index and  rename the Series name as “Amount”, meanwhile give the name of the “None” index as “Product Desc” to make it more meaningful. E.g.:

df.set_index(["Salesman", "Order Date"])\
    .stack()\
    .reset_index(name='Amount')\
    .rename(columns={'level_2':'Product Desc'})

With the above code, you can see the output similar to below:

pandas convert columns to rows, wide to long format, pandas melt

 

If you do not want to have the 0 sales amount records, you can easily apply a filter to the DataFrame to have cleaner data.

Using Pandas Melt method

The melt method is a very powerful function to unpivot data from wide to long format. It is like the opposite operation to the pivot_table function, so if you are familiar with pivot_table function or the Excel pivot table, you shall be able to understand the parameters easily.

To achieve the same result as per the stack function, we can use the below code with melt method:

df.melt(id_vars=['Salesman', 'Order Date'], 
        value_vars=['Beer', 'Red Wine', 'Whisky', 'White Wine'],
        var_name="Product Desc",
        value_name='Amount')

The id_vars specifies the columns for grouping rows. The value_vars and var_name specify the columns to unpivot and the new column name, and the value_name indicates the name of the value column. To help you better understand this parameters, you can imagine how the data is generated via pivot table in Excel, now it’s the reversing process.

pandas convert columns to rows, wide to long format, pandas melt

 

Using Pandas wide_to_long Method

The wide_to_long method is quite self-explanatory by its name. The method uses pandas.melt under the hood, and it is designed to solve some particular problems. For instance, if your columns names follows certain patterns such as including a year or number or date, you can specify the pattern and extract the info when converting those columns to rows.

Below is the code that generates the same output as our previous examples:

pd.wide_to_long(
    df, 
    stubnames="Amount", 
    i=["Salesman", "Order Date"], 
    j="Product Desc", 
    suffix=r"|Red Wine|White Wine|Whisky|Beer").reset_index()

The stubnames parameter specifies the columns for the values converted from the wide format. And i specifies the columns for grouping the rows, and j is the new column name those stacked columns. Since our product column names does not follow any pattern, in the suffix parameter, we just list out all the product names.

As the wide_to_long returns a MultiIndex DataFrame, we need to reset index to make it flat data structure.

You may not see the power of this function from the above example, but if you look at the below example from its official document, you would understand how wonderful this function is when solving this type of problems.

pandas convert columns to rows, wide to long format, pandas melt, pandas wide_to_long

Performance Consideration

When testing the code performance for the above 3 methods, the wide_to_long method would take significant longer time than the other two methods, and melt seems to be the fastest. But the result may vary for large set of data, so you will need to evaluate again based on your data set.

#timeit for stack method
4.52 ms ± 329 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

#timeit for melt method
3.5 ms ± 238 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

#timeit for wide_to_long method
17.8 ms ± 709 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

Conclusion

In this article, we have reviewed through 3 pandas methods to convert columns to rows when you need to unpivot your data or transform it from wide to long format for further analysis. A simple testing shows that melt method performs the best and the wide_to_long takes the longest time, but bear in mind that wide_to_long method has its specific use cases which the other functions may not be able to achieve.

 

Manipulate Audio File in Python, pydub, download youtube,cut video python

Manipulate Audio File in Python With 6 Powerful Tips

Introduction

Dealing with audio files may not be that common to a Python enthusiast, but sometimes you may wonder if you are able to manipulate audio files in Python for your personal interest. For instance, if you really like some music, and you want to edit some parts of it and save into your phone, so that you can listen it during your study or outdoor exercise without skipping those annoying ads.

In this post, I would be introducing you a simple yet useful library for you to manipulate audio file in Python code.

Prerequisites:

You need to install Pydub in your working environment, below is the installation command via pip (click for more tips):

pip install pydub

The library has the dependency to ffmpeg in order to support most of the audio file formats, so you may use the below command to install if you do not have it yet:

pip install ffmpeg

Download Video from YouTube

As I am going to use a funny video from YouTube for the later demonstration, I would need install another library – youtube_dl to download the video into my local folder:

pip install youtube_dl

Below is the command to download the video from YouTube with the given URL and the output file name. You can also use -f to specify the file format if the original video has multiple format:

youtube-dl "https://www.youtube.com/watch?v=Zo6F_qtQCCc" -o "hongshaorou.mp4"

You may see the below output messages from your terminal, and the final output file will be saved to your current directory:

Manipulate Audio File in Python, pydub, download youtube,cut video python

Now let’s import pydub and use this video to explore what we can do with this library.

from pydub import AudioSegment
import os

Extract Sound From A Video File

To load a video file, we can use the from_file function from the AudioSegment module:

base_dir = r"c:\sounds"
sound = AudioSegment.from_file(os.path.join(base_dir, "hongshaorou.mp4"))

There are also other functions such as from_mp3, from_wav or from_ogg etc., depending on what type of audio files you want to read. With the export function, you can easily convert the video file into another format:

sound.export(os.path.join(base_dir, "hsr.mp3"), format="mp3")

There are some more parameters you can use to specify the metadata when you save the file, e.g.:

sound.export(os.path.join(base_dir, "hsr.mp3"),
                           format="mp3",
                           bitrate="192k",
                           tags={"album": "chinese cuisine", "artist": "not sure"},
                           cover= os.path.join(base_dir,"hongshaorou.jpeg"))

And you can also retrieve the meta info as per below:

from pydub.utils import mediainfo
mediainfo('hsr3.mp3')

Split/Cut Audio Clips

With the AudioSegment object, you can cut the audio file like slicing a list by specifying the starting point and ending point in milliseconds. For instance, to cut our audio file from 1:18 to 1:33 and save it to mp3:

first_cut_point = (1*60 + 18) * 1000
last_cut_point = (1*60 + 33) * 1000

sound_clip = sound[first_cut_point:last_cut_point]

sound_clip.export(os.path.join(base_dir, "hsr.mp3"), format="mp3")

Increase/Reduce Sound Volume

You can make the sound louder or quieter by adding/subtracting the decibels as per below:

#increase volume by 10dB for the first 2 seconds
sound_clip_1 = sound_clip[:2000] + 10

#reduce volume by 5dB for the last 3 seconds
sound_clip_2 = sound_clip[-3000:] - 5

#combine multiple sound clips
final_clip = sound_clip_1 + sound_clip[2000:-3000] + sound_clip_2

Play Sound In Python

If you are running the code in Jupyter Lab, you can simply execute the final_clip and see how the result sounds like:

Otherwise you use the playback module to play the sound as per below:

from pydub.playback import play
play(final_clip)

Adding Silence In The Sound

Silence can be added to your sound clip as per below:

#Adding 1 second silence before the sound clip
AudioSegment.silent(duration=1000) + sound_clip[:5000]

Overlay Audio Onto Another Audio

The overlay function allows you to overlay one AudioSegment to another AudioSegment object. For instance:

sound_clip[5000:10000].overlay(final_clip[:5000])

There are some more useful functions for editing audio files, you can see full API document from here.

Conclusion

In this article, we have reviewed through a few very useful functions in the pydub library which allows you to manipulate audio file such as converting audio formats, combining, splitting or editing sound clips. With these tips, you shall be able to create your own sound clips in a few lines of Python code. In this post, we have also used the youtube-dl library which allows you to download the video from YouTube and some other video streaming website. You may refer to this reddit discussion if you are wondering whether this is legal. But I believe it should be alright if you just use for your personal exploration on the Python programming.

Python generate QR code, Python read QR code, Photo by Lukas on Unsplash

Read and Generate QR Code With 5 Lines of Python Code

 Introduction

QR Code is the most popular 2 dimensional barcodes that widely used for document management, track and trace in supply chain and logistics industry, mobile payment,  and even the “touchless” health declaration and contact tracing during the COVID-19 pandemic. Comparing to 1D barcode, QR code can be very small in size but hold more information, and also easier for scanning as you can scan it from any direction.

In this article, I would be sharing with you how to use some pure Python packages to generate QR code and read QR code from images.

Generate QR code with Python

To generate QR code, we will use a Python package called qrcode. Below is the pip command to install this package:

#install qrcode together with pillow
pip install qrcode[pil]

#or install qrcode if you already have pillow installed
pip install qrcode

As it has dependency to Pillow package, you will need to have this package installed as well. Once you have these packages ready, let’s import the modules at the beginning of our code:

import qrcode
from PIL import Image

Generating a QR code with this qrcode library can be easily done with 1 line of code:

img = qrcode.make('QR Code')

If you check the “img” object from Jupyter Notebook, you can see the below image:

Generate QR code Python, Read QR code Python

This make function provides a quick way to generate QR code with all the default parameters. To specify the parameters like the size, style or border of the boxes, you can use the QRCode class. For instance:

qr = qrcode.QRCode(
    version=1,
    error_correction=qrcode.constants.ERROR_CORRECT_L,
    box_size=10,
    border=4,
)

Here is the explanations for these parameters:

version – QR code has 40 different sizes which indicated as the version parameter in above, version 1 represents a 21×21 matrix.  You can use (v-1)*4 + 21 to calculate the size of the matrix for each of the version number.

error_correctionspecifies error correction level which controls how many error correction code blocks to be inserted in order to achieve the error correction capability. In another words, if you want your barcode to be readable even when it’s damaged (or have a logo/image onto it) , you may increase the error correction level, but this would also make your barcode more compact.

box_size the number of pixels of the square box

border – the thickness of the square box border

Once you have a QRCode instance, you can use the below code to specify the barcode data, color and generate a image:

#barcode content
qr.add_data('codeforests.com')

#auto adjust the size
qr.make(fit=True)

#specifying barcode color
img = qr.make_image(fill_color="#040359", back_color="#f7f7fa")

If you check the “img” object from Jupyter Notebook again, you shall see something similar to below:

 

Generate QR code Python, Read QR code Python

 

To use the same barcode style to generate new barcode, you can just clear the data and then re-generate a new image object:

 

qr.clear()
qr.add_data('Python Tutorials')

img2 = qr.make_image(fill_color="#015b82", back_color="TransParent")

When inspecting the “img2” in Jupyter Notebook, you shall see below:

Generate QR code Python, Read QR code Python, Python QR code with different color

 

You can simply use the “save” method to save it into an image file since it is a Pillow Image object:

img2.save("qr_code.png")

The qrcode package cannot directly generate multiple QR codes into one image, if you need that, you may use the Pillow package to combine the images. For instance:

#create a blank image
new_img = Image.new("RGBA", (600, 350), "#fcfcfc")

new_img.paste(img, (0, 0))

new_img.paste(img2, (300, 0))

new_img.save("multi-QR-code.png")

The above will create a new image and combine the two barcode images into one. If you check the saved image file, you shall see:

 

Generate QR code Python, Read QR code Python, generate multiple QR codes on one page

With this package, you can also generate styled QR code e.g.: rounded corners, radial gradient, embedded image or different color masks. You can take a look at the samples from it’s office site.

Read QR Code in Python

To read QR code, we will use another Python package called pyzbar. You can use below pip command to install it:

pip install pyzbar

This library is also a very easy to use, you can directly pass in a Pillow Image object, numpy.ndarray or raw bytes to the decode method to detect the barcode. For instance:

import pyzbar.pyzbar as pyzbar
from pyzbar.pyzbar import ZBarSymbol

input_image = Image.open("multi-QR-code.png")

decoded_objects = pyzbar.decode(input_image, symbols=[ZBarSymbol.QRCODE])

The decode method returns a list of barcode objects detected from the image with their position info. You can use the symbols parameter to restrict what type of barcodes you want to detect. When this parameter is not specified, all its supported barcode types will be checked.

From the above, you can further loop through the list to get the actual content data of the barcodes:

for obj in decoded_objects:
    zbarData = obj.data.decode("utf-8")
    print(zbarData)

You shall see the below result:

Generate QR code Python, Read QR code Python

In your real-world project, if you need to read one barcode among the multiple barcodes from a document, you may try to use the symbols to restrict the barcode types, or use regular expression to validate the detected barcode data in order to find the correct one you need.

If you need to do a lot of image pre-processing or even read barcode from video or webcam, you may install OpenCV and use the detectAndDecodeMulti method to read the QR code.

Conclusion

In this article, we have reviewed through two simple but useful packages – qrcode for generating QR code, and pyzbar for reading the content from a QR code. There are quite many other Python packages for generating all sorts of one or two dimensional barcodes, some are in pure Python packages and some are Python wrappers, you may take a look at the summary table from this blog if any specific barcode type you need is not supported by these two packages.