Resolved: Use tqdm progress bar with asyncio

Question:

I am trying to create a single progress bar that will be updated whenever an async task is done.
I have the following code

scan_results = []
tasks = [self.run_scan(i) for i in input_paths]

pbar = tqdm(total=len(tasks), desc='Scanning files')

for f in asyncio.as_completed(tasks):
    value = await f
    pbar.update(1)
    scan_results.append(value)

The above code generates a single progress bar as but it is not updated until all tasks are finished (it shows either 0% or 100% while there’s more than a single task)
I also tried using tqdm.asyncio.tqdm.gather

with tqdm(total=len(input_paths)):
     scan_results = await tqdm.gather(*[self.run_scan(i) for i in input_paths])

The above code generates multiple progress bars and as in the previous code block, it shows either 0% or 100%
My starting point was

scan_results = await asyncio.gather(*[self.run_scan(i)for i in input_paths])

Would appreciate your help on making it work with a single and dynamic progress bar

Answer:

If you call self.pbar.update(1) inside the run_scan scan method after creating concurrent tasks, each task will update the pbar for self. So your class should look like the following

class Cls:
    async def run_scan(self, path):
        ...
        self.pbar.update(1)

    def scan(self, input_paths):
        loop = asyncio.get_event_loop()
        tasks = [loop.create_task(self.run_scan(i)) for i in input_paths]
        self.pbar = tqdm(total=len(input_paths), desc='Scanning files')
        loop.run_until_complete(asyncio.gather(*tasks))
        loop.close()

If you have better answer, please add a comment about this, thank you!

Resolved: Use tqdm progress bar with asyncio

Resolved: Shopware 400 Status Error “This value is too long. It should have 255 character or less.” When I Try Updating Database Table

Resolved: Using AWK to count multiple patterns in a file

Resolved: significance letter above bar graphic in wrong order

Leave A Reply