BentoML (bundled Gradio app, multipart boundary handling): Appending a long run of characters to a multipart boundary

Impact

Appending a long run of characters to a multipart boundary makes the server chew through each one, burning CPU until the model endpoint stops answering. An unauthenticated request takes a served model offline, and on a GPU node the pod keeps holding its device allocation while it is useless.

Who can reach it

Any unauthenticated client that can reach the BentoML serving endpoint. Single crafted HTTP request, no session.

What to do

Upgrade BentoML past 1.4.5 and restart the serving pods. Put a request-size and rate limit in front of the model endpoint, and set CPU limits on the serving container so the abuse cannot spread to co-located pods.

References