Args:
calib_size (int, optional): Size of calibration dataset. Defaults to 1024.
recommended to increase for production runs (e.g. 1536), or decrease e.g. to 256 for quick testing.
calib_dataset (str, optional): Hugginface dataset to use for calibration. Defaults to 'cnn_dailymail'.
uses split='train' and quantized based on 'text' column.
calib_max_seq_length (int, optional): Maximum sequence length for calibration. Defaults to 2048.
Configuration for quantization of TRT models
Args: calib_size (int, optional): Size of calibration dataset. Defaults to 1024. recommended to increase for production runs (e.g. 1536), or decrease e.g. to 256 for quick testing. calib_dataset (str, optional): Hugginface dataset to use for calibration. Defaults to 'cnn_dailymail'. uses split='train' and quantized based on 'text' column. calib_max_seq_length (int, optional): Maximum sequence length for calibration. Defaults to 2048.