Developer Tooling
Latency & Token Metrics
Precision tracking of token usage and response times.
Orbyt measures and displays critical performance metrics for every single execution:
- TTFT (Time To First Token): The latency between sending the request and receiving the first streaming chunk.
- Total Latency: The end-to-end time taken for the entire request.
- Prompt Tokens: The number of tokens sent in the initial request.
- Completion Tokens: The total number of tokens generated by the model.
- Total Usage: The sum of prompt and completion tokens, used for budget tracking.