ken

python brownie network setup, brownie network configuration, brownie networks, brownie hardhat, brownie infura

All You Need to Know About Python Brownie Network Setup

Introduction

In my previous posts, I have reviewed through how to use Python Brownie to deploy smart contract and interact with a live contract by loading from ABI file or a block explorer. Brownie provides a list of predefined network configurations which probably will be sufficient for you most of the time, but It also gives you flexibility to configure and work on your own customized networks.

In this article, I will walk through with you a few tips on the Python Brownie network setup.

Prerequisite

You will need to install the latest Brownie package in your Python virtual environment if you have not done yet so. Below is the pip command:

pip install eth-brownie

By default, Brownie uses Ganache for the development network. But if you plan to use hardhat with Brownie, you may also install hardhat. Read further from more details.

Brownie Check Existing Networks

To check the list of networks currently available, you can use the below command:

brownie networks list

And you shall see a long list of networks that Brownie supported by default, below is the first few:

python brownie network setup, brownie network configuration, brownie networks, brownie hardhat

Almost all the EVM compatible networks are included as well as the other networks which provided public JSON-RPC endpoint and block explorer.

To check the network details, such as the chain ID, host or block explorer etc, you can set the verbose option as true. For instance:

brownie networks list true

You shall see the network details similar to below:

python brownie network setup, brownie network configuration, brownie networks, brownie hardhat, brownie infura

This basically outputs all the information in the network-config.yaml file which is located under your useraccount/.brownie folder. And this setup can be shared among multiple of your brownie projects.

Brownie Add New Network

There are a few ways to add a new network to Brownie config, you can either do it with command line or through the configuration file.

Adding Network from Command Line

When you want to add a public network, Brownie requires at least network id, host and chainid. The rest are optional, but you may still need them, for instance, you will need to specify the block explorer if you want to get your source code verified by block explorer when deploying your contract.

Brownie uses the network id to lookup for the configurations, so it shall be unique among all the networks you have configured. For instance, you can configure as much networks as you can with different ids for Ethereum mainnet. And below is an example to add a network by using Alchemy API to access the Ethereum mainnet:

brownie networks add Ethereum mainnet-alchemy chainid=1 explorer=https://api.etherscan.io/api host=https://eth-mainnet.alchemyapi.io/v2/$ALCHEMY_PROJECT_TOKEN_ID multicall2=0x5BA1e12693Dc8F9c48aAD8770482f4739bEeD696 name="Mainnet (alchemy)"

You shall see below new network added successfully:

python brownie network setup, brownie network configuration, brownie networks, brownie alchemy api

When adding a development network, you must specify the environment as development and provide two additional fields – cmd and cmd_settings. This would let Brownie knows how to launch the local network and connect to it.

Brownie Import Network Configuration File

When you have a network YAML file shared from someone else, you can easily load it into your Brownie network with the import command. E.g. below is a yaml file with the network details:

live:
- name: Ethereum
  networks:
  - chainid: 1
    explorer: https://api.etherscan.io/api
    host: https://eth-mainnet.alchemyapi.io/v2/$ALCHEMY_PROJECT_TOKEN_ID
    id: mainnet-alchemy
    multicall2: '0x5BA1e12693Dc8F9c48aAD8770482f4739bEeD696'
    name: Mainnet (alchemy)

We can save it to network-config.yaml to your current folder and run below command:

brownie networks import ./network-config.yaml

You shall see below output from your terminal:

Brownie v1.17.0 - Python development framework for Ethereum

SUCCESS: Network settings imported from 'network-config.yaml'

If you check network list again, you shall see the newly added network:

python brownie network setup, brownie network configuration, brownie networks, brownie import network

You will encounter error if the id already exists, but if you intend to replace the original setup, you can add a replace option when importing the configuration. E.g.:

brownie networks import ./network-config.yaml true

As I mentioned earlier that Brownie actually saved the network configurations under the .brownie folder. You can also add new network directly into the file, but I would not recommend this approach just to avoid messing up the entire Brownie environment.

Modify Existing Network

Brownie also allows to modify your existing network from the command line. For instance, you can change the network details by using below:

brownie networks modify mainnet-alchemy name=mainnet-alchemy

Output as per below:

python brownie network setup, brownie network configuration, brownie networks, brownie modify network

With the modify option, you can also update the other attributes such as the host such as to switch the network provider from Infura to Alchemy or Pocket Network etc.

Delete Network

Deleting an existing network can be done easily with below command:

brownie networks delete mainnet-alchemy

You shall get below output if Brownie found the network id from your existing configuration:

SUCCESS: Network 'mainnet-alchemy' has been deleted

Brownie Export Network Config

The network details can also be exported from Brownie command line. You can use the below command to export it to your current folder:

brownie networks export .

This would generate below outputs and you shall see a yaml file created with all the network details:

Brownie v1.17.0 - Python development framework for Ethereum

SUCCESS: Network settings exported as 'network-config.yaml'

This would be very helpful when you want to share your network setup to someone else.

Using Hardhat with Brownie

Hardhat is an Ethereum development environment with similar functionality to Brownie. It includes a hardhat network which is similar to Ganache, and Brownie added the support for hardhat network in the recent release.

To use hardhat, you shall follow this installation guide to install nodejs and hardhat. And then launch your hardhat node as per below:

npx hardhat node --port 8545

With the above command, it shall start a local server at below port:

Started HTTP and WebSocket JSON-RPC server at http://127.0.0.1:8545/

And from another terminal window, you can use below command:

brownie console --network hardhat

Brownie will connect to the hardhat RPC server, and now you can do your development and testing same as what you do in Ganache. E.g. To deploy a smart contract with the hardhat default accounts:

python brownie network setup, brownie network configuration, brownie networks, brownie hardhat deploy contract

From the Hardhat terminal, you shall also see some output as per below:

python brownie network setup, brownie network configuration, brownie networks, brownie hardhat console

Conclusion

In this post, we have reviewed through a few commands that Brownie provided for managing the network configurations such as adding new networks, modifying or delete existing networks as well as import/export network configuration files. One thing to be noted is that the Brownie project configuration brownie-config.yaml also allows you to specify a networks section, but this is only meant for specifying the default network for the current project, anything else you have defined under this section would not overwrite the Brownie network configuration.

Follow me on twitter if you have any feedback!

Python interact with smart contract, python brownie, python mint NFT, python call smart contract function

Python Interact with Smart Contract

In the previous post, we have discussed how to deploy a NFT smart contract with Python Brownie. Once your contract is deployed into the blockchain, you can interact with your live contract either by building up your own Dapp or using scripts to perform some repetitive tasks like batch token transferring or airdrop. In this article, we will share with you some popular approaches to interact with smart contract in Python once the contract is live in public network.

Prerequisite

If you do not have Brownie installed, you shall go ahead to run the below pip command in your own Python virtual environment to install it:

#run in your venv
pip install eth-brownie

Brownie is heavily relying on Web3.py and use ganache cli as the default emulator for the development network, so you will get these dependencies installed automatically.

And assuming you have already compiled your smart contract and run the deployment script without any issue, if not, you may check through this post before proceeding. Now let’s explore how to interact with a deployed contract.

Work with Contract in Ganache Local Network

Using Ganache CLI

The Ganache CLI is used by default for development, and the local network is spin up when you are running your deployment script and terminated as soon as the script finished execution. In this case, the deployed contract is not stored anywhere permanently and you can only interact with it within same session before Ganache is shut down. The deployed contract is added into Brownie’s deployment map automatically, so you do not have to note down contract address when you want to refer to it. For instance, you can get your last deployed contract as per below:

# assuming a smart contract LegendNFT has been deployed
last_deployed_contract = LegendNFT[-1]

I would recommend you to use Brownie console when you just get started. It’s a Python interactive mode with all Brownie stuff loaded, and you can immediately get feedbacks if something goes wrong in your code. When everything is tested ok in the console, you can then copy your code to the Python script and make it re-usable for later use.

To start Brownie console for your local network, you can use the below command:

brownie console

And from the console, you can also deploy your contract:

>>> contract = LegendNFT.deploy({'from' : accounts[0]})

Transaction sent: 0x38643bde3b513dbaedf887c60162e5bb915b32bde2a804172dfaa2520b4d4ac8
Gas price: 0.0 gwei Gas limit: 12000000 Nonce: 0
LegendNFT.constructor confirmed Block: 1 Gas used: 3188048 (26.57%)
LegendNFT deployed at: 0x3194cBDC3dbcd3E11a07892e7bA5c3394048Cc87

Then you can verify that the contract address returned from deploy function is the same as the last record stored in Brownie’s deployment map:

>>> LegendNFT[-1]
<LegendNFT Contract '0x3194cBDC3dbcd3E11a07892e7bA5c3394048Cc87'>
>>> contract == LegendNFT[-1]
True

In this case, you can always use LegendNFT[-1] to get the last deployed contract instead of replying on the contract address returned from your deploy function, and the deployment script can be kept separately so that you don’t accidently redeploying the contract when you want to run your other functions.

One advantage of using console is that you can use dir function to inspect what are the available functions in the contract instance, e.g.:

>>> dir(LegendNFT[-1])
[abi, address, approve, balance, balanceOf, bytecode, decode_input, getApproved, 
getTotalNFTsMintedSoFar, get_method, get_method_object, info, isApprovedForAll, 
makeLegendNFT, name, ownerOf, safeTransferFrom, selectors, setApprovalForAll, 
signatures, supportsInterface, symbol, tokenURI, topics, totalSupply, transferFrom, tx]

Some of these functions are not explicated implemented in the contract but inherited from other contracts, you will be able to call them as long as the functions have public or external visibility specifier.

Now you can call whatever functions available in your contract, for instance:

>>> contract.name()
'Codeforests Legend'
>>> contract.totalSupply()
1000000

>>> contract.getTotalNFTsMintedSoFar()
0

And you can also call your mint function which will change the state of the blockchain:

>>> contract.makeLegendNFT({'from':accounts[0]})

Transaction sent: 0x6dabe95e4c54d43864e91bfeb398c2bfb0549bf48d51db2cd42b9882e501a905
  Gas price: 0.0 gwei   Gas limit: 12000000   Nonce: 1
  LegendNFT.makeLegendNFT confirmed   Block: 2   Gas used: 777946 (6.48%)

<Transaction '0x6dabe95e4c54d43864e91bfeb398c2bfb0549bf48d51db2cd42b9882e501a905'>

Of course you would still not be able to see how it looks like for your newly minted NFT since the minting was only happening in your local network. If you use the tokenURI function and pass in the first token ID, you shall see some base64 encoded data similar to below:

>>> contract.tokenURI(0)

'data:application/json;base64,eyJuYW1lIjogIlNwbGVuZGlkIEMjIFByb2dyYW0iLCAiZGVzY3JpcHRpb24iOiAiQSBsZWdlbmQgTkZUIGNvbGxlY3Rpb24gZm9yIGFsbCBjb2RlZm9yZXN0cyByZWFkZXJzLiIsICJpbWFnZSI6ICJkYXRhOmltYWdlL3N2Zyt4bWw7YmFzZTY0LFBITjJaeUI0Yld4dWN6MG5hSFIwY0RvdkwzZDNkeTUzTXk1dmNtY3ZNakF3TUM5emRtY25JSEJ5WlhObGNuWmxRWE53WldOMFVtRjBhVzg5SjNoTmFXNVpUV2x1SUcxbFpYUW5JSFpwWlhkQ2IzZzlKekFnTUNBek5UQWdNelV3Sno0OGMzUjViR1UrTG1KaGMyVWdleUJtYVd4c09pQWpSa1ZHUlVaRk95Qm1iMjUwTFdaaGJXbHNlVG9nVEdWaFozVmxJRk53WVhKMFlXNDdJR1p2Ym5RdGMybDZaVG9nTWpSd2VEc2dmVHd2YzNSNWJHVStQSEpsWTNRZ2QybGtkR2c5SnpFd01DVW5JR2hsYVdkb2REMG5NVEF3SlNjZ1ptbHNiRDBuSXpBMU1UWXlNaWNnTHo0OGRHVjRkQ0I0UFNjMU1DVW5JSGs5SnpVd0pTY2dZMnhoYzNNOUoySmhjMlVuSUdSdmJXbHVZVzUwTFdKaGMyVnNhVzVsUFNkdGFXUmtiR1VuSUhSbGVIUXRZVzVqYUc5eVBTZHRhV1JrYkdVblBsTndiR1Z1Wkdsa0lFTWpJRkJ5YjJkeVlXMDhMM1JsZUhRK1BDOXpkbWMrIn0='

This is the URI info for your NFT, and you can use base64 decoder to decode the data to see more details, or use this website to preview the NFT content.

Using Ganache UI

If you are using Ganache desktop version, you can add a new network in Brownie and let Brownie to connect to your Ganache RPC client. Below is the command to add a new network:

brownie networks add Development ganache-local host=http://127.0.0.1 network_id=5777 port=7545 cmd=ganache-cli

The network ID/host/port can be found from your Ganache client as per below:

Python Interact with Smart Contract, Python mint NFT, Python Brownie, Ganache client

Once you have added the network, you can start your Brownie console as per below:

brownie console --network ganache-local

#Attached to local RPC client listening at '127.0.0.1:7545'...

And then deploy your contract:

>>> LegendNFT.deploy({'from' : accounts[0]})

You shall see some transactions went through if you check the Ganache client:

Python Interact with Smart Contract, Python mint NFT, Python Brownie, Ganache client

So you will be able to trace the transaction history from Ganache client, and once you restart it, those history data will be gone and you will get a fresh new blockchain environment same as how the Ganache CLI works.

Work with Smart Contract in Public Network

When you use Brownie to deploy your contracts to public network such as rinkeby or mainnet, it automatically organize the contract ABI file by chain ID into the build/deployments folder as per below:

Python Interact with Smart Contract, Python mint NFT, Python Brownie, Ganache client

So as long as you do not delete these files, you can use ContractName[-1] to get the last deployed contract. To connect to rinkeby in Brownie console, specify the network as rinkeby:

brownie console --network rinkeby

Then you can interact with your contract same as working in the local network except you will need to load a real wallet. If you want to be sure you are working on the correct contract, you can use LegendNFT.at with deployed contract address instead of LegendNFT[-1] :

Python Interact with Smart Contract, Python mint NFT, Python Brownie, Ganache client

Awesome! you can now mint your NFT from your deployed contract in a public network. Probably you don’t want to always type the same code in the console window when you want to mint a new NFT. so let’s put the code we have tested into a script called mint.py in our scripts folder as per below:

from brownie import network, config, accounts, LegendNFT

def get_account():
    if network.show_active() in ["development", "ganache-local"]:
        return accounts[0]
    else:
        return accounts.add(config["wallets"]["from_key"])

def mint_nft():
    contract = LegendNFT[-1]
    account = get_account()
    tx = contract.makeLegendNFT({"from" : account})
    tx.wait(1)
    print("Minted a new NFT with txn hash:", tx.txid)

def main():
    mint_nft()

So when we are running this script in local network, it will just pick a dummy account for deployment, but when working in public network, it will load the account from our config for deployment.

Let’s run this script with below:

brownie run mint.py --network rinkeby

You shall see similar output as we did from console window:

Python Interact with Smart Contract, Python mint NFT, Python Brownie, Ganache client

And here is the transaction details we can see from the etherscan by searching the transaction hash.

What if I have accidentally deleted my ABI file or I want to access the contract created by someone else where I do not have the ABI data?

Don’t worried, as long as you have your contract address, you will still be able to access the contract. You can use the below from_explorer from the Contract object to fetch the information from network explorer. For instance:

>>> contract = Contract.from_explorer("0xaa629be88190fe2077b970aa02f52da40ce98454")
Fetching source of 0xaA629BE88190fe2077b970aa02f52dA40cE98454 from api-rinkeby.etherscan.io...

>>> contract.name()
'Codeforests Legend'

And if you want to try the hard way, you can actually copy the ABI data from the network explorer such as etherscan, ployscan etc. into a local file, then load it manually into a JSON object. For instance, I have saved the ABI data from here into a file called abi_from_rinkeby.json:

Python Interact with Smart Contract, Python mint NFT, Python Brownie, Smart Contract ABI

and with Brownie console connected to Rinkeby,  I can access my contract as per below:

>>> import json
>>> with open("./scripts/abi_from_rinkeby.json") as file:
...     contract_abi = json.load(file)
...
>>> contract = Contract.from_abi("LegendNFT", "0xaA629BE88190fe2077b970aa02f52dA40cE98454", contract_abi)
>>>
>>> contract.totalSupply()
1000000
>>>
>>> contract.getTotalNFTsMintedSoFar()
3

Now with the above approach, you can start mint an NFT from someone else’s contract without going through their website.

Conclusion

In this article, we have reviewed through a few approaches you can use to interact with your smart contract. Brownie is built up on top of the Web3.py, but you can still use the APIs from the web3 module to interact with your contract. There are also quite a few of popular JavaScript libraries such as web3.js, ethers.js etc., you may take a look if you are planning to interact with the contract from your frontend JavaScript code.

brownie deploy smart contract python, NFT smart contract, mint NFT, free NFT

Deploy Your First NFT with Python

Introduction

NFT (Non-Fungible Token) has exploded since last year and keeps on roaring regardless of how the cryptocurrency is performing. You must have heard of CryptoPunk, BAYC or the most recent Phantom Bear if you are following any crypto news. In the nutshell, NFT is a smart contract with some meta data such as image, video or some other attribute data to make it unique from one another. And since it’s on blockchain, it’s easy for anybody to verify the ownership. In this article, I will walk you through the process on how to deploy your own NFT with Python and we will be using a Python based smart contract development framework called Brownie.

Prerequisite

You will need to install the Brownie package. It is recommended to use pipx to install it in an isolated environment, but since I have my own virtual environment created, I would continue to use pip instead. Below is the pip command to install Brownie:

#run in your venv
pip install eth-brownie

In order to test your code locally, Brownie will automatically install ganache cli which launches a local Ethereum blockchain for you to execute your smart contracts. You can follow its documentation to download and install it if you wish to install it manually.

Get Started with Brownie

Let’s create an empty folder and then run the below command to initiate a new Brownie project:

brownie init

With the above command, brownie will create a project structure for you under your current folder. The majority of your work shall be done in the following folders:

  • contracts – for solidity smart contract source code
  • interfaces – for the interface files that referenced by the smart contract
  • scripts – for deployment scripts or interacting with deployed contracts
  • tests – for test scripts

 

Brownie supports both Solidity and Vyper language for creating smart contracts. In this article, I will be using the smart contract in Solidity style as the example.

Create NFT Smart Contract

NFTs use ERC721 token standard, so we will need to create a smart contract that implements this standard. Since the objective of this article is to walk through the process rather than a deep dive into the smart contract coding, I have just shared a sample smart contract here.  You can download this LegendNFT.sol smart contract file and the libraries folder, then put them under the contracts folder.

Here is a quick explanation of what this smart contract does:

  • an ERC721 contract with name Codeforests Legend, token symbol CFL and total supply of 1,000,000
  • 3 lists of words to generate a “random” word phrase
  • a SVG xml template to create a picture of the above word phrase
  • a list of color codes to provide “random” background color to the SVG
  • a mint function for minting the NFT tokens
  • finally a function to return how many tokens have minted

 

As you can see from the source code that I used two external contracts from OpenZeppelin, so I will need to specify this dependency so that they can be recognized when compiling the contract. Brownie provides a config file brownie-config.yaml which allows you specify these configurations. This file is not created automatically when initiating the project as all the settings are optional, you will only need it when necessary.

To specify the dependencies, you can manually create this yaml file under your project root folder. And add the below configurations:

dependencies:
  - OpenZeppelin/openzeppelin-contracts@4.4.1
compiler:
  solc:
    remappings:
      - '@openzeppelin=OpenZeppelin/openzeppelin-contracts@4.4.1'

It basically tells Brownie package manager to automatically download this package for you and let compiler find the correct path of the package when using the short form ‘import @openzeppelin/…’.

Compile Smart Contract

Once you have saved your contract and the config file, you can run below compile command on your terminal:

brownie compile

You shall see brownie auto downloaded the dependencies and compiled the code:

brownie compile smart contract, NFT smart contract

The output Json files will be placed into the build/contracts folder. The next step we shall try to deploy this contract locally.

Deploy Smart Contract Locally

To deploy our contract locally, we will need to use the ganache as I mentioned in the beginning. By default, when you want to deploy a contract, brownie will always start the ganache local network and deploy it to the local network. Brownie provides a console window where you can use the commands to deploy contract, but since we may need to do the deployment multiples for testing or in different networks, so let’s create a re-usable Python script called deploy.py under the scripts folder for the deployment purpose.

When your smart contract is compiled, the contract class object will be automatically added to brownie runtime environment, so we can import it from brownie directly.

Below is the Python code for deploying my LegendNFT contract:

from brownie import LegendNFT, network, config, accounts


def deploy_contract():
    account = accounts[0]
    legend_contract = LegendNFT.deploy({"from" : account})
    print("contract has been deployed successfully to :", legend_contract.address)

    return legend_contract

def main():
    deploy_contract()

The accounts is some dummy accounts provided by ganache only for your local network, so we just pick the first dummy account as the contract creator for the from parameter.

Note that your script needs to have a main function to let brownie know which function to run.

Next, let’s run the below command in the terminal:

brownie run .\scripts\deploy.py

You shall see something similar to the below output showing that both contract address and transaction hash have been generated:

brownie deploy smart contract python, NFT smart contract

Congratulations that you’ve just deployed your contract to your local network, although you may have no idea if the contract works correctly. Don’t worry, let’s continue to explore how to deploy to a public network and come back to the unit test later.

Deploy Smart Contract to Public Network

The public network usually refers to the public accessible and persistent blockchain network such as the Ethereum mainnet, rinkeby, kovan, popsten etc. To access the public network, you will need to connect to a node, but it would be too much to run your own node just for deploying a contract.

Luckily there are some service providers like Infura, Alchemy or Pocket Network etc., who allows you to use their API to connect to the public network nodes. For Brownie, it uses Infura as the default provider for connecting to public network. If you run below command in your terminal:

brownie networks list true

You can see it is using Infura’s API:

brownie deploy smart contract python, NFT smart contract, rinkeby network

In this case, you will need to sign up Infura for a free account and get your own project API token. Once you have your API token, you can create a environment variable called WEB3_INFURA_PROJECT_ID and assign your token ID to it. My personal preference would be putting this information into a .env file and load it into the Brownie config file.

And another thing you need is a real wallet account with some fund, as deployment will always incur some cost for confirming the transaction. You can create a wallet with Metamask, and for testnet like rinkeby, you can get some testing ether token from chainlink rinkeby faucet. You can also find other faucets for the rest of the testnets.

So let’s put below info in the .env file:

PRIVATE_KEY = 0x{your wallet private key}
WEB3_INFURA_PROJECT_ID = {your infrua project ID}
ETHERSCAN_TOKEN = {etherscan API token; to submit your contract source code to etherscan (optional)}

Note: to add 0x before your wallet private key so that it’s recognized as hex string. If you wish to get your contract source code verified by etherscan, you will need to sign up with etherscan and get an API token as well.

And add below lines in the brownie-config.yaml file:

dotenv: .env

wallets:
  from_key: ${PRIVATE_KEY}

# optional for verifying the contract source code
networks:
  development:
    verify : false
  rinkeby:
    verify : true

I strongly recommend you to add .env in .gitignore, so that you won’t accidently push your sensitive information into github. Your wallet private key for testnet and mainnet is the same, so bear in mind to not expose to anyone.

Now let’s also do some minor change to our deployment script, so that it uses the new account we created for deploying the contract to public network.

def deploy_contract():
    if(network.show_active() == "development"):
        account = accounts[0]
    else:
        account = accounts.add(config["wallets"]["from_key"])

    legend_contract = LegendNFT.deploy(
        {"from" : account}, 
        publish_source=config["networks"][network.show_active()].get("verify")
    )
    print("contract has been deployed successfully to :", legend_contract.address)

    return legend_contract

We have added a condition to check whether we are working on local/development network, when it’s not local network, we will load our account by the private key and use it to sign the transaction for deployment.

Now we have everything ready, let’s run the deploy.py again and specify the target network with the –network option, e.g. :

brownie run .\scripts\deploy.py --network rinkeby

Below is the output from my terminal:

brownie deploy smart contract python, NFT smart contract, Python deploy smart contract to public network

It would take slightly longer time since it requires the transaction to be minted and confirmed. You can copy the contract address or transaction hash then search it from etherscan rinkeby network.

You can see my deployed contract address here and the verified source code here.

Awesome! You just had your contract deployed in the public network where everybody can see it!!! Since you’ve not yet minted any NFT from your contract, you won’t see anything visually at the moment. To give you a idea of how your NFTs will look like, I have created this website for minting the LegendNFT, you can try and experience the minting process and then view the NFT from Opeasea. It currently supports Rinkeby, Matic and Matic Mumbai network. Below is the minting page after you have connected your wallet:

brownie deploy smart contract python, NFT smart contract, mint NFT, free NFT

Conclusion

Creating your own NFT can be a very fun journey although you may not be able to eventually sell it on the market. And there are just too much details to be covered and which is impossible to write in one post, so I will try to write a few more articles on how to perform unit test, interact with your deployed contract, as well as deploy the contract to other EVM compatible networks. Do follow me on my twitter and let me know if you encounter any issue with your contract deployment.

pandas convert columns to rows, convert wide to long, pandas melt

Pandas Tips – Convert Columns To Rows

  Introduction

In one of my previous posts – Pandas tricks to split one row of data into multiple rows, we have discussed a solution to split the summary data from one row into multiple rows in order to standardize the data for further analysis. Similarly, there are many scenarios that we have the aggregated data like a Excel pivot table, and we need to unpivot it from wide to long format for better analysis. In this article, I will be sharing with you a few tips to convert columns to rows with pandas DataFrame.

Prerequisites

To run the later code examples, you shall get pandas installed in your working environment. Below is the pip command to install pandas:

pip install pandas

And we will be using the data from this file for the later demonstration, so you may download and examine how the data looks like with below code:

import pandas as pd
import os
data_dir = "c:\\your_download_dir"
df = pd.read_excel(os.path.join(data_dir, "Sample-Data.xlsx"))

You shall see the sample sales data as per below:

pandas convert columns to rows, wide to long format, pandas melt

The sales amount has been summarized by each product in the last 4 columns. With this wide data format, it would be difficult for us to do some analysis, for instance, the top salesman by month by products or the best seller products by month etc.

A better data format should be transforming the product columns into rows so that each single row only represents 1 product and its sales amount. Now let’s start to explore what are the different ways to convert columns to rows with pandas.

Using Pandas Stack Method

The most immediate solution you may think of would be using the stack method as it allows you to stack the columns vertically onto each other and make it into multiple rows.  For our case, we will need to specify the DataFrame index as “Salesman” and “Order Date“, so that the product columns will stack based on this index. For instance:

df.set_index(["Salesman", "Order Date"]).stack()

If you check the result now, you shall see the below output:

pandas convert columns to rows, wide to long format, pandas melt

This is an MultiIndex Series with index name – [‘Salesman’, ‘Order Date’, None], so you can reset the index and  rename the Series name as “Amount”, meanwhile give the name of the “None” index as “Product Desc” to make it more meaningful. E.g.:

df.set_index(["Salesman", "Order Date"])\
    .stack()\
    .reset_index(name='Amount')\
    .rename(columns={'level_2':'Product Desc'})

With the above code, you can see the output similar to below:

pandas convert columns to rows, wide to long format, pandas melt

 

If you do not want to have the 0 sales amount records, you can easily apply a filter to the DataFrame to have cleaner data.

Using Pandas Melt method

The melt method is a very powerful function to unpivot data from wide to long format. It is like the opposite operation to the pivot_table function, so if you are familiar with pivot_table function or the Excel pivot table, you shall be able to understand the parameters easily.

To achieve the same result as per the stack function, we can use the below code with melt method:

df.melt(id_vars=['Salesman', 'Order Date'], 
        value_vars=['Beer', 'Red Wine', 'Whisky', 'White Wine'],
        var_name="Product Desc",
        value_name='Amount')

The id_vars specifies the columns for grouping rows. The value_vars and var_name specify the columns to unpivot and the new column name, and the value_name indicates the name of the value column. To help you better understand this parameters, you can imagine how the data is generated via pivot table in Excel, now it’s the reversing process.

pandas convert columns to rows, wide to long format, pandas melt

 

Using Pandas wide_to_long Method

The wide_to_long method is quite self-explanatory by its name. The method uses pandas.melt under the hood, and it is designed to solve some particular problems. For instance, if your columns names follows certain patterns such as including a year or number or date, you can specify the pattern and extract the info when converting those columns to rows.

Below is the code that generates the same output as our previous examples:

pd.wide_to_long(
    df, 
    stubnames="Amount", 
    i=["Salesman", "Order Date"], 
    j="Product Desc", 
    suffix=r"|Red Wine|White Wine|Whisky|Beer").reset_index()

The stubnames parameter specifies the columns for the values converted from the wide format. And i specifies the columns for grouping the rows, and j is the new column name those stacked columns. Since our product column names does not follow any pattern, in the suffix parameter, we just list out all the product names.

As the wide_to_long returns a MultiIndex DataFrame, we need to reset index to make it flat data structure.

You may not see the power of this function from the above example, but if you look at the below example from its official document, you would understand how wonderful this function is when solving this type of problems.

pandas convert columns to rows, wide to long format, pandas melt, pandas wide_to_long

Performance Consideration

When testing the code performance for the above 3 methods, the wide_to_long method would take significant longer time than the other two methods, and melt seems to be the fastest. But the result may vary for large set of data, so you will need to evaluate again based on your data set.

#timeit for stack method
4.52 ms ± 329 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

#timeit for melt method
3.5 ms ± 238 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

#timeit for wide_to_long method
17.8 ms ± 709 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

Conclusion

In this article, we have reviewed through 3 pandas methods to convert columns to rows when you need to unpivot your data or transform it from wide to long format for further analysis. A simple testing shows that melt method performs the best and the wide_to_long takes the longest time, but bear in mind that wide_to_long method has its specific use cases which the other functions may not be able to achieve.

 

Manipulate Audio File in Python, pydub, download youtube,cut video python

Manipulate Audio File in Python With 6 Powerful Tips

Introduction

Dealing with audio files may not be that common to a Python enthusiast, but sometimes you may wonder if you are able to manipulate audio files in Python for your personal interest. For instance, if you really like some music, and you want to edit some parts of it and save into your phone, so that you can listen it during your study or outdoor exercise without skipping those annoying ads.

In this post, I would be introducing you a simple yet useful library for you to manipulate audio file in Python code.

Prerequisites:

You need to install Pydub in your working environment, below is the installation command via pip (click for more tips):

pip install pydub

The library has the dependency to ffmpeg in order to support most of the audio file formats, so you may use the below command to install if you do not have it yet:

pip install ffmpeg

Download Video from YouTube

As I am going to use a funny video from YouTube for the later demonstration, I would need install another library – youtube_dl to download the video into my local folder:

pip install youtube_dl

Below is the command to download the video from YouTube with the given URL and the output file name. You can also use -f to specify the file format if the original video has multiple format:

youtube-dl "https://www.youtube.com/watch?v=Zo6F_qtQCCc" -o "hongshaorou.mp4"

You may see the below output messages from your terminal, and the final output file will be saved to your current directory:

Manipulate Audio File in Python, pydub, download youtube,cut video python

Now let’s import pydub and use this video to explore what we can do with this library.

from pydub import AudioSegment
import os

Extract Sound From A Video File

To load a video file, we can use the from_file function from the AudioSegment module:

base_dir = r"c:\sounds"
sound = AudioSegment.from_file(os.path.join(base_dir, "hongshaorou.mp4"))

There are also other functions such as from_mp3, from_wav or from_ogg etc., depending on what type of audio files you want to read. With the export function, you can easily convert the video file into another format:

sound.export(os.path.join(base_dir, "hsr.mp3"), format="mp3")

There are some more parameters you can use to specify the metadata when you save the file, e.g.:

sound.export(os.path.join(base_dir, "hsr.mp3"),
                           format="mp3",
                           bitrate="192k",
                           tags={"album": "chinese cuisine", "artist": "not sure"},
                           cover= os.path.join(base_dir,"hongshaorou.jpeg"))

And you can also retrieve the meta info as per below:

from pydub.utils import mediainfo
mediainfo('hsr3.mp3')

Split/Cut Audio Clips

With the AudioSegment object, you can cut the audio file like slicing a list by specifying the starting point and ending point in milliseconds. For instance, to cut our audio file from 1:18 to 1:33 and save it to mp3:

first_cut_point = (1*60 + 18) * 1000
last_cut_point = (1*60 + 33) * 1000

sound_clip = sound[first_cut_point:last_cut_point]

sound_clip.export(os.path.join(base_dir, "hsr.mp3"), format="mp3")

Increase/Reduce Sound Volume

You can make the sound louder or quieter by adding/subtracting the decibels as per below:

#increase volume by 10dB for the first 2 seconds
sound_clip_1 = sound_clip[:2000] + 10

#reduce volume by 5dB for the last 3 seconds
sound_clip_2 = sound_clip[-3000:] - 5

#combine multiple sound clips
final_clip = sound_clip_1 + sound_clip[2000:-3000] + sound_clip_2

Play Sound In Python

If you are running the code in Jupyter Lab, you can simply execute the final_clip and see how the result sounds like:

Otherwise you use the playback module to play the sound as per below:

from pydub.playback import play
play(final_clip)

Adding Silence In The Sound

Silence can be added to your sound clip as per below:

#Adding 1 second silence before the sound clip
AudioSegment.silent(duration=1000) + sound_clip[:5000]

Overlay Audio Onto Another Audio

The overlay function allows you to overlay one AudioSegment to another AudioSegment object. For instance:

sound_clip[5000:10000].overlay(final_clip[:5000])

There are some more useful functions for editing audio files, you can see full API document from here.

Conclusion

In this article, we have reviewed through a few very useful functions in the pydub library which allows you to manipulate audio file such as converting audio formats, combining, splitting or editing sound clips. With these tips, you shall be able to create your own sound clips in a few lines of Python code. In this post, we have also used the youtube-dl library which allows you to download the video from YouTube and some other video streaming website. You may refer to this reddit discussion if you are wondering whether this is legal. But I believe it should be alright if you just use for your personal exploration on the Python programming.