feat: add GLM-5 FP8 SGLang benchmark for MI355X

[functionstackx]

Add single-node benchmark configuration for GLM-5 FP8 on MI355X:

• Config key: glm5-fp8-mi355x-sglang
• Model: zai-org/GLM-5-FP8 with NSA tilelang backends
• Image: rocm/sgl-dev:v0.5.8.post1-rocm720-mi35x-20260219
• TP=8, concurrency 4-64 for 1k1k, 1k8k, and 8k1k

Closes #761

Generated with Claude Code

e2e Tests: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/22581147775?target=https://github.com

[functionstackx]

@claude there is an transformer dependency error

can u add this to the glm5 benchmark script

pip install git+https://github.com/huggingface/transformers.git

following anush's gist recipe

Traceback (most recent call last):
  File "/home/cameronamd@semianalysis.com/.local/bin/hf", line 3, in <module>
    from huggingface_hub.cli.hf import main
ModuleNotFoundError: No module named 'huggingface_hub'
[aiter] import [module_aiter_enum] under /sgl-workspace/aiter/aiter/jit/module_aiter_enum.so
[2026-02-19 21:52:22] INFO core.py:501: import [module_aiter_enum] under /sgl-workspace/aiter/aiter/jit/module_aiter_enum.so
Traceback (most recent call last):
  File "/opt/venv/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1360, in from_pretrained
    config_class = CONFIG_MAPPING[config_dict["model_type"]]
  File "/opt/venv/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1048, in __getitem__
    raise KeyError(key)
KeyError: 'glm_moe_dsa'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/sgl-workspace/sglang/python/sglang/launch_server.py", line 32, in <module>
    server_args = prepare_server_args(sys.argv[1:])
  File "/sgl-workspace/sglang/python/sglang/srt/server_args.py", line 5591, in prepare_server_args
    return ServerArgs.from_cli_args(raw_args)
  File "/sgl-workspace/sglang/python/sglang/srt/server_args.py", line 5077, in from_cli_args
    return cls(**{attr: getattr(args, attr) for attr in attrs})
  File "<string>", line 331, in __init__
  File "/sgl-workspace/sglang/python/sglang/srt/server_args.py", line 733, in __post_init__
    self._handle_gpu_memory_settings(gpu_mem)
  File "/sgl-workspace/sglang/python/sglang/srt/server_args.py", line 1010, in _handle_gpu_memory_settings
    if not self.use_mla_backend():
  File "/sgl-workspace/sglang/python/sglang/srt/server_args.py", line 5110, in use_mla_backend
    model_config = self.get_model_config()
  File "/sgl-workspace/sglang/python/sglang/srt/server_args.py", line 5091, in get_model_config
    self.model_config = ModelConfig.from_server_args(self)
  File "/sgl-workspace/sglang/python/sglang/srt/configs/model_config.py", line 250, in from_server_args
    return ModelConfig(
  File "/sgl-workspace/sglang/python/sglang/srt/configs/model_config.py", line 127, in __init__
    self.hf_config = get_config(
  File "/sgl-workspace/sglang/python/sglang/srt/utils/common.py", line 3475, in wrapper
    result = func(*args, **kwargs)
  File "/sgl-workspace/sglang/python/sglang/srt/utils/hf_transformers_utils.py", line 320, in get_config
    raise e
  File "/sgl-workspace/sglang/python/sglang/srt/utils/hf_transformers_utils.py", line 315, in get_config
    config = AutoConfig.from_pretrained(
  File "/opt/venv/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1362, in from_pretrained
    raise ValueError(
ValueError: The checkpoint you are trying to load has model type `glm_moe_dsa` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+[https://github.com/huggingface/transformers.git`](https://github.com/huggingface/transformers.git%60)?target=https://github.com
Server died before becoming healthy. Exiting.
srun: error: mia1-p01-g12: task 0: Exited with exit code 1
+ scancel 3420
+ ls 'gpucore.*'
Waiting for result file... (attempt 1)
Waiting for result file... (attempt 2)
Waiting for result file... (attempt 3)
Waiting for result file... (attempt 4)
Waiting for result file... (attempt 5)
Waiting for result file... (attempt 6)
Waiting for result file... (attempt 7)
Waiting for result file... (attempt 8)
Waiting for result file... (attempt 9)

[Klaud-Cold]

Claude finished @functionstackx's task in 39s —— View job

---

Adding transformers dependency fix to GLM-5 benchmark script

• Read the current benchmark script
• Add pip install git+https://github.com/huggingface/transformers.git to the script
• Commit and push the change

Added pip install git+https://github.com/huggingface/transformers.git before the hf download step in benchmarks/glm5_fp8_mi355x.sh. The container's bundled transformers version doesn't recognize the glm_moe_dsa model type that GLM-5 uses, so installing from source provides the latest model type support.

View commit ·

[functionstackx]

@claude move glm5_fp8_mi355x.sh to under benchmarks/single_node and look at the refactor on how to fix it and look at other sglang single node mi355 scripts on how to do it after the refactor benchmarks/single_node/

[Klaud-Cold]

Claude finished @functionstackx's task in 1m 57s —— View job

---

Moving GLM-5 benchmark to single_node directory

• Read current benchmarks/glm5_fp8_mi355x.sh
• Explore benchmarks/single_node/ for other SGLang MI355X scripts
• Understand the refactored pattern (runner resolves path as benchmarks/single_node/)
• Move script to benchmarks/single_node/ with corrected source path
• Commit and push changes

Moved benchmarks/glm5_fp8_mi355x.sh → benchmarks/single_node/glm5_fp8_mi355x.sh and updated the source path from ./benchmark_lib.sh to ../benchmark_lib.sh to match the directory depth used by all other single-node scripts. The runner (launch_mi355x-amds.sh:184) constructs the script path as benchmarks/single_node/{prefix}_{precision}_mi355x.sh, so this is required for the benchmark to actually execute.

View commit ·

[chunfangamd]

Looks good to me