What can Python Click do for you? At the end of this guide, you’ll be able to create your own command line interfaces that can pass context, with commands that can take mandatory or optional arguments, and nested commands. This example Python project will show you how to interact with different click commands from the building blocks such as click.command to the more advanced commands such as click.make_pass_decorator. If you have any questions please don’t hesitate to reach out to me @yujian_tang. You can find the source code here.
Click, or “Command Line Interface Creation Kit” is a Python library for building command line interfaces. The three main points of Python Click are arbitrary nesting of commands, automatic help page generation, and supporting lazy loading of subcommands at runtime. Click provides their own rationalization as to why you should use Python Click over other Python CLI libraries. My reasoning, as a developer, for picking Click is that it’s easy to use and it provides the necessary capabilities to create complex command line interfaces.
Let’s get started, the first thing we’re going to do is create a group, in click, a group is a set of (supposedly) related commands. This group that we’re going to make is going to be based around interacting with a JSON document, you can either download this from the link, or follow the tutorial towards the bottom of the page that will show you how I generated it using AssemblyAI, which we’ll load as a dictionary. All of the commands in this group will interact with our loaded dictionary, so we will use the pass_context decorator to keep the dictionary stored in context. The context is an object that holds all the context of the execution of a command. In a group command, the context object is used to pass information between commands.
import click
import json
'''
@click.group(<name>) creates a command that instantiates a group class
a group is intended to be a set of related commands
@click.argument(<argument name>) tells us that we will be passing an argument
and referring to that argument in the function by the name we pass it
@click.pass_context tells the group command that we're going to be using
the context, the context is not visible to the command unless we pass this
In our example we'll name our group "cli"
'''
@click.group("cli")
@click.pass_context
@click.argument("document")
def cli(ctx, document):
"""An example CLI for interfacing with a document"""
_stream = open(document)
_dict = json.load(_stream)
_stream.close()
ctx.obj = _dict
def main():
cli(prog_name="cli")
if __name__ == '__main__':
main()Upon creation of our group and declaring it in main, we should have an automatically generated "--help" option, like so:
Yours shouldn’t have all the commands listed yet, but that’s just a sneak peak at what we’re going to build. NOTE - all commands must be defined between main() and our click.group decorator.
Now that we’ve created our first group in Python Click, let’s add our first command to it. Let’s make a command that will check the context for us to confirm that we have the right type. Earlier we said that we want to load our object as a dictionary into our context, so we should expect a result like:
All we have to do to make this command a part of the group we created earlier is decorate our function with the decorator <group name>.command(<command name>). To make sure that this object is able to access context, we also use the pass_context decorator.
Passing Context in Python Click
import pprint
'''
@click.command(<name>) creates a command that can be called with
the name that we pass through
Here we'll create an example command that prints out our context object,
which we expect to be a json looking dictionary
'''
@cli.command("check_context_object")
@click.pass_context
def check_context(ctx):
pprint.pprint(type(ctx.obj))Now that we have some familiarity with decorators and Python Click, let’s use Click’s functionality to make our own pass decorator. Why would we make our own? So that we can pass a specific type of object through (later we’ll also demonstrate how to use pass_obj). Let’s make a pass decorator to specifically pass a dictionary object through, we'll call it pass_dict.
'''
Here we'll make a pass decorator, which we can use to pass
the last object stored of a type of our choosing in the context
by using click.make_pass_decorator(<type>)
'''
pass_dict = click.make_pass_decorator(dict)Let’s make a command that will use the pass_dict decorator to pass in the dictionary object that is stored. The Python Click library will search for the innermost object (in context) of the type we specified above (dict) and pass that to our function. When we call our get_keys function, it should look like this:
All we’re going to do is pass the dictionary in using the pass_dict decorator and then extract the keys and use Click’s echo, style, and secho commands to print our keys out in different colors.
'''
click.echo is click's version of the echo command
click.style lets us style our output
click.secho is a command that takes a message, and a style command,
is a combination of click.echo and click.style
This command returns the keys to our dictionary object and
demonstrates how to use click.echo, click.style, and click.secho
'''
@cli.command("get_keys")
@pass_dict
def get_keys(_dict):
keys = list(_dict.keys())
click.secho("The keys in our dictionary are", fg="green")
click.echo(click.style(keys, fg="blue"))Okay, now that we know what keys are in our dictionary object, let’s create a command that will get the value to a key that we pass. We’re also going to decorate this with pass_context because we’re going to use this command to demonstrate one of the three tenants of click - arbitrary nesting of commands. We’ll create a second command after get_key that will get the value of the summary key by invoking the get_key command.
'''
This command gets a specific key from the context object
'''
@cli.command("get_key")
@click.argument("key")
@click.pass_context
def get_key(ctx, key):
pprint.pprint(ctx.obj[key])
'''
click.invoke(<command>, <args>) is click's way of letting us
arbitrarily nest commands. NOTE: this command can only be used
when both the command being invoked AND the the command
doing the invoking use @click.pass_context
Since we already have a get_key command, we can just call that
to print out a summary
'''
@cli.command("get_summary")
@click.pass_context
def get_summary(ctx):
ctx.invoke(get_key, key="summary")Both should result in the same output as shown:
Adding Optional Parameters to Your Command Line Interface
Now let’s go through how to use the click.options decorator. This decorator is used when you want to pass optional parameters to your commands. It can be used to pass an argument, or to be passed as a flag. If you want your option to be passed as a flag, set the is_flag parameter to True when using your decorator.
'''
@click.option(<one dash usage>, <two dash usage>, is_flag (optional), help = <help>)
is how we can pass options to our command
We'll create a function that gets the "results" of our dictionary
and we will pass it two optional arguments, one to specify that
we want a specific key from the results, and a flag to indicate
whether or not we want to save our results to a json file
'''
@cli.command("get_results")
@click.option("-d", "--download", is_flag=True, help="Pass to download the result to a json file")
@click.option("-k", "--key", help="Pass a key to specify that key from the results")
@click.pass_context
def get_results(ctx, download: bool, key: str):
results = ctx.obj['results']
if key is not None:
result = {}
for entry in results:
if key in entry:
if key in result:
result[key] += entry[key]
else:
result[key] = entry[key]
results = result
if download:
if key is not None:
filename = key+'.json'
else:
filename = "results.json"
with open(filename, 'w') as w:
w.write(json.dumps(results))
print("File saved to", filename)
else:
pprint.pprint(results)In our example, we’re displaying the "results" key of our dictionary. You’ll notice that I also passed in the --key option that we coded up above and chose to get the text of our results.
If I also pass in -d or --download, the command will download the file to a .json document instead of displaying it in the terminal.
Okay, the last click decorator that I want to introduce is pass_obj which acts just as we would expect it to act based on our experience making a pass decorator earlier and pass_context with Python Click earlier. We’ll also do something fun with our text, we can grab our text in three different formats. As one big text block, which I will set as the default, by sentence, or by paragraph.
'''
@click.pass_obj is similar to @click.pass_context, instead
of passing the whole context, it only passes context.obj
We'll do something fun with our text extractor, we'll include
options to extract as either paragraphs or sentences, and
default to returning one big block of text
'''
@cli.command("get_text")
@click.option("-s", "--sentences", is_flag=True, help="Pass to return sentences")
@click.option("-p", "--paragraphs", is_flag=True, help="Pass to return paragraphs")
@click.option("-d", "--download", is_flag=True, help="Download as a json file")
@click.pass_obj
def get_text(_dict, sentences, paragraphs, download):
"""Returns the text as sentences, paragraphs, or one block by default"""
results = _dict['results']
text = {}
for idx, entry in enumerate(results):
if paragraphs:
text[idx] = entry['text']
else:
if 'text' in text:
text['text'] += entry['text']
else:
text['text'] = entry['text']
if sentences:
sentences = text['text'].split('.')
for i in range(len(sentences)):
if sentences[i] != '':
text[i] = sentences[i]
del text['text']
pprint.pprint(text)
if download:
if paragraphs:
filename = "paragraphs.json"
elif sentences:
filename = "sentences.json"
else:
filename = "text.json"
with open(filename, 'w') as w:
w.write(json.dumps(results))
print("File saved to", filename)This is what the responses should look like:
Sentences:
Paragraphs:
No optional parameters:
Alright now I’m going to demonstrate how to have two different Click command groups running from the same file. What we’re going to do is allow users to pass either a JSON file structured in the way our example JSON file is, or any .mp3 file. To follow the rest of this guide, you’ll need an API key from AssemblyAI. What we’re going to do is go over how to do exactly what we did with the text option above, but we’ll get our sentences and paragraphs directly from AssemblyAI.
For just an example of what it looks like to have two python click command groups running, skip down to the last code segment.
The first thing we’ll do is declare some global variables and our imports. Notice that I’m importing my auth key from a configure file. It’s recommended that you store your API keys in a separate file to avoid uploading and exposing them. We’ll need requests to send requests, sleep from time to automatically poll our endpoint, and auth key from configure to access the AssemblyAI API.
import requests
from time import sleep
from configure import auth_key
transcript_endpoint = "https://api.assemblyai.com/v2/transcript"
upload_endpoint = 'https://api.assemblyai.com/v2/upload'
headers = {
"authorization": auth_key,
"content-type": "application/json"
}
CHUNK_SIZE = 5242880Let’s create our second group. I’m going to name this one "assembly" since we’re using it to interact with AssemblyAI’s API. I’m going to have the function that creates the group also take our mp3 file, upload and transcribe it, and save the returned id to context. We’ll use the id later to easily get both sentences and paragraphs from an API endpoint.
@click.group("assembly")
@click.pass_context
@click.argument("location")
def assembly(ctx, location):
"""A CLI for interacting with AssemblyAI"""
def read_file(location):
with open(location, 'rb') as _file:
while True:
data = _file.read(CHUNK_SIZE)
if not data:
break
yield data
upload_response = requests.post(
upload_endpoint,
headers=headers, data=read_file(location)
)
audio_url = upload_response.json()['upload_url']
print('Uploaded to', audio_url)
transcript_request = {
'audio_url': audio_url,
'iab_categories': 'True',
}
transcript_response = requests.post(transcript_endpoint, json=transcript_request, headers=headers)
transcript_id = transcript_response.json()['id']
polling_endpoint = transcript_endpoint + "/" + transcript_id
print("Transcribing at", polling_endpoint)
polling_response = requests.get(polling_endpoint, headers=headers)
while polling_response.json()['status'] != 'completed':
sleep(30)
print("Transcript processing ...")
try:
polling_response = requests.get(polling_endpoint, headers=headers)
except:
print("Expected to wait 30 percent of the length of your video")
print("After wait time is up, call poll with id", transcript_id)
return transcript_id
categories_filename = transcript_id + '_categories.json'
with open(categories_filename, 'w') as f:
f.write(json.dumps(polling_response.json()['iab_categories_result']))
print('Categories saved to', categories_filename)
ctx.obj = polling_response.json()['id']Thanks to the brilliant design of AssemblyAI’s API endpoints, we can have the functions to get paragraphs or sentences be almost the exact same function. All we need to do is construct the endpoint, and then send an HTTP GET request.
@assembly.command("get_sentences")
@click.pass_context
def get_sentences(ctx):
sentences_endpoint = transcript_endpoint + "/" + ctx.obj + "/sentences"
sentences_response = requests.get(sentences_endpoint, headers=headers)
pprint.pprint(sentences_response.json())
@assembly.command("get_paragraphs")
@click.pass_context
def get_paragraphs(ctx):
paragraphs_endpoint = transcript_endpoint + "/" + ctx.obj + "/paragraphs"
paragraphs_response = requests.get(paragraphs_endpoint, headers=headers)
pprint.pprint(paragraphs_response.json())Alright now that we’ve got a couple example commands in our assembly group, let's cover how to actually have both command groups running at the same time and run a different group “intelligently” (based on our input). What we’re going to do is convert our main() function to check if our first argument contains a JSON file or an .mp3 file.
import sys
def main():
if ".json" in sys.argv[1]:
cli(prog_name="cli")
if ".mp3" in sys.argv[1]:
assembly(prog_name="assembly")As you can see from the two images below, our new command line tool can recognize input and run the commands correctly from that.
Now we’re at the end of our tutorial on how to use Python Click to make a command line interface. If you have any questions regarding how to use Click, tweet me @yujian_tang or reach out to me on LinkedIn. For more information on how to use the AssemblyAI API, check out this tutorial to do Python Speech Recognition in Under 25 Lines or this tutorial on building a YouTube Transcriber and follow us @assemblyai
