In this post, we will see how to resolve Logs csv to JSON or any mechanism to get metrics
Question:i have a log file and i need to extract stats based on logs captured.
Sample log format:
I need to get
- Total data, conversion time and process time to complete and total time taken.
- Min data, Max data, Avg data — respective conversion time taken for min data, max data and Avg data(for suppose there are 10 iteration – in that min data in that 10 iterations of data, max data in that 10 iterations of data and Average of 10 iterations and their respective conversion, process and time taken)
I started converting csv to json first and thought of extracting from there but am stuck to move further. here is what i tired,
Best Answer:You can use pandas to read your csv and pivot the segments corresponding to entries then concat:
time takento the output, you can sum all column elements:
If you have better answer, please add a comment about this, thank you!