Inferless Inc.

Inferless Inc. provides serverless GPU inference services designed for spiky and unpredictable workloads. The company offers the fastest serverless GPU inference with low cold-start times and allows scaling from a single user to billions, with payment only for usage. Users can deploy models from Hugging Face, Git, Docker, or CLI, and benefit from automatic redeploy options for quick shipping. The platform features an in-house load balancer for automatic scaling, customizable containers for specific software and dependencies, and NFS-like writable volumes supporting simultaneous connections to various replicas. Automated CI/CD for models eliminates the need for manual re-imports, and detailed call and build logs aid in efficient model monitoring and refinement. Dynamic batching increases throughput by enabling server-side request combining, and endpoints can be customized with settings for scale down, timeout, concurrency, testing, and webhooks. Inferless Inc. has achieved SOC 2, ISO 27001, and GDPR compliance and offers a pay-per-second billing model for GPU usage, with a free trial providing 10 hours of free credit and no credit card required. The platform supports Nvidia T4, A10, and A100 GPUs for inference workloads and provides both shared and dedicated GPU instances with different pricing tiers. Users can dynamically scale models to reduce fixed costs significantly and integrate into staging environments within hours. Separate environments for production, non-production, and development are available at no additional cost, and startups receive a $30 free credit to kickstart their compute journey. Inferless Inc. supports large custom models up to 16GB, with assistance available for larger models, and ensures data security with isolated execution environments and AES-256 encryption for model storage. The company offers detailed pricing information with per-second billing and no upfront costs, enterprise-level security with regular vulnerability scans and penetration testing, and supports various machine learning applications, including computer vision, NLP, recommendations, and scientific computing. Users can access the SSH terminal through a web portal and deploy machine learning models on serverless GPUs in minutes. The platform uses a proprietary algorithm to balance autoscaling and latency, offers end-to-end model deployment with automatic endpoint creation and monitoring data, and provides a developer-friendly usage-based billing module. Inferless Inc. supports deployment of models from Hugging Face, Sagemaker, Pytorch, and Tensorflow, and reduces model cold start times by 99%. The platform supports both custom and pre-trained machine learning models, offers a community feature for model-sharing and collaboration, and provides an open-source library called COG for deploying models. The Bring Your Own Container (BYOC) approach allows for deploying container-based GPU instances, and fully managed and scalable AI endpoints cater to diverse workloads and applications. The platform fee is $12.99/month with per millisecond billing, and a summary matrix compares various serverless GPU platforms. Inferless Inc. has locations in the United States and India and offers a pricing model starting from $0.33/hr.