The Modelbit Changelog

Week ending April 26, 2024

🛠️ Improvements:

• Added VRAM to the resource graphs in Training Jobs.
• Added queue time to performance traces.
• Added a "copy" button to the source code viewer.
• Made various visual improvements to the registry UX, especially when comparing multiple models.
• Added the modelbit.create_branch Python API method.
• Updated the logs viewer to show much longer log lines in a collapsed-but-expandable fashion.
• Made it clearer on the command line when you need to upgrade your modelbit python package.

🐛 Bug Fixes:

• Fixed a bug where creating a new branch from an individual deployment page led to a 404.
• Fixed a bug where the model registry page could return a 500 error if the registry is empty.
• Fixed a race condition that could cause new-deployment health checks to respond with a "Version not found" error.

Week ending April 19, 2024

When branching your git repo, Modelbit no longer automatically deploys all pre-existing deployments on the new branch. Those deployments now show a "Deploy to this branch" button on the new branch.

🛠️ More Improvements:

• Added support for custom tags in the DataDog integration. Also lower-cased our DataDog service name to modelbit.
• Made improvements to the UIs for adding secrets, rate limits and SSH keys for the first time.
• Added validation when passing directory names as optional parameters to modelbit clone.
• Made some performance improvements to the Modelbit registry.
• Added an error icon to erroring deployments on the deployment index page.
• Added a loading animation to currently-deploying deployments on the deployment index page.

🐛 Bug Fixes:

• Fixed a bug where SSH keys could fail to add but appear to be successfully added.
• Implemented the recommended workaround to a known Nvidia container crash bug.
• Fixed a race condition that could cause containers to fail to attach to available GPUs depending on boot order.

Week ending April 12, 2024

Introducing billing alerts! Get notified when your usage exceeds daily, weekly or monthly thresholds.

🛠️ More Improvements:

• Modelbit no longer shows secrets and third-party integrations' API keys in the web app once they've been created.
• Made various performance improvements to the Modelbit web application.

🐛 Bug Fixes:

• Fixed a bug where runtime tracing could crash Python 3.6 deployments.
• Added some comments to Modelbit-generated code from notebook deployments to make it clearer which lines of code were generated by Modelbit.
• Fixed a bug where there's a gap in performance traces for deployments that don't import modelbit.

Week ending April 5, 2024

Introducing performance tracing! Analyze your model's performance in Modelbit, including custom telemetry. Learn more.

🛠️ More Improvements:

• Added cloudpickle support to the model registry for models that don't work well with ordinary pickle! Learn more.
• Added some comments to Modelbit-generated code from notebook deployments to make it clearer which lines of code were generated by Modelbit.
• Made various deploy-time and cold-start-time performance improvements.

🐛 Bug Fixes:

• Fixed some UX bugs that can crop up when multiple git branches are deleted in quick succession.
• Fixed the UI colors in the billing pop-up in dark mode.
• Fixed a bug where the helpful error messages for missing packages did not trigger for models loaded from the registry.
• Fixed a race condition that could cause new deployments at new workspaces to silently fail some of the time.

Week ending March 29, 2024

🛠️ Improvements:

• Modelbit now attempts to validate the presence of required files in each deployment directory client-side before accepting git commits.
• Added an integration for Anthropic. Learn more here.

🐛 Bug Fixes:

• Fixed a bug where very long conditionals in requirements.txt's could make the Environment page unusable.
• Fixed a bug where training jobs could fail to start when created in new environments.

Week ending March 22, 2024

🛠️ Improvements:

• Added per-deployment and system-wide health checks for connectivity to CUDA runtimes.
• Added tooltips to deployment status icons.
• Modelbit now sets YOLO_CONFIG_DIR to silence warnings during YOLO deployments.
• Modelbit now cleans up old Snowflake inference functions when their corresponding deployments are deleted in Modelbit.
• Improved request queueing logic to make better use of Modelbit's parallelism for slow deployments.
• Updated the logic for auto-created workspace names to more cleanly handle ".com.region" TLDs.
• Made some performance improvements to the model registry when adding many objects at a time.

🐛 Bug Fixes:

• Fixed a bug where very long mock Snowflake return values caused the API Endpoints screen to be unusable.
• Fixed a bug where the python_packages parameter of modelbit.deploy() could fail to correctly parse environment markers.
• Fixed a bug where XGBoost was sometimes not automatically detected when nested inside SKLearn objects.
• Fixed a bug where hover effects didn't trigger on folders in the source code viewer in dark mode.

Week ending March 15, 2024

🛠️ Improvements:

• Added the ability to disable retry-on-timeout logic for async inference callbacks. Learn more.
• Made Modelbit's notebook output easier to read in Hex dark mode.
• Made performance improvements to Modelbit's git backend server and web application.
• Made various performance improvements to Snowpark deployments.

🐛 Bug Fixes:

• Fixed a bug where the sample code for creating Snowpark configurations was missing a use database command.
• Fixed a bug where creating an account directly from a notebook login would lead to an error.
• Fixed a bug where slow Snowpark deployments could error nondeterministically due to threading issues.

Week ending March 8, 2024

Modelbit now deploys directly into Snowflake compute with Snowpark Python UDFs!

🛠️ Improvements:

• Added new Python APIs modelbit.get_snowflake_mock_return_value(...) and modelbit.set_snowflake_mock_return_value(...) to access and modify the Snowflake mock return values for already-deployed models!
• Added a Python API to add metrics to an already-added model in the registry! Learn more.
• Modelbit no longer automatically archives old, un-called deployments, as this was causing more problems than it was solving.
• With the power of colors and text styles, made it clearer that the authentication link output by  modelbit clone is in fact a link.
• Added tooltips for the emojis that signify special deployment properties like GPU usage and rate limits.
• Made training job error messages more helpful.
• Made the login-from-notebook page much prettier.
• Made some performance improvements to modelbit.deploy() from a notebook.

🐛 Bug Fixes:

• Fixed a bug where otherwise-successful training jobs could occasionally fail for no reason at the last minute.
• Fixed an off-by-one-day bug in attributing compute minute consumption to prepaid annual plans. No customer bills were impacted.
• Fixed a bug where archived deployments could re-create their Snowflake functions.
• Fixed some grammar in Modelbit Python API outputs.
• Fixed a bug where Snowpark inference logs time traveled into the future. Reports that Snowpark itself contains time travel functionality are unverified at this time.

Week ending March 1, 2024

🛠️ Improvements:

• Made usage charts clearer when the "latest" version has changed over time.
• Added the ability to set the return value of Snowflake mock functions. Learn more.

🐛 Bug Fixes:

• Fixed a bug where inference log lines with errors incorrectly reported their batch size as zero.
• Fixed some incorrect syntax highlighting in private package names.
• Fixed a bug where "Deploying this model" could show for deployments that only have training jobs.

Week ending February 23, 2024

Updated deployments' API Endpoints screen to include the Python API!

🛠️ Improvements:

• Improved the performance of Modelbit datasets when used as a feature store in production.
• Improved Python environment detection for YOLO models.
• Added the ability to set the return value of Snowflake mock functions. Learn more.
• Made additional improvements to GPU fallback logic.

🐛 Bug Fixes:

• Fixed a bug where Modelbit would sometimes try to pickle `None` when deploying from notebooks.
• Fixed a bug where partner integrations would sometimes fail to apply themselves to running deployments.
• Fixed a bug where Modelbit would sometimes 500 in scenarios with large volumes of successive batches.
• Fixed a bug where Modelbit would sometimes rebuild environments unnecessarily in between training and deploying the same model.
• Fixed a bug where the start time of a currently-running training job could time travel into the future.
• Fixed a bug where only a subset of large Redshift batches would complete.

Week ending February 16, 2024

Modelbit now supports rate limiting! You can create custom rate limits and apply them to various deployments to throttle their execution speed.

🛠️ Improvements:

• System metrics have been added to our status page! You can now verify Modelbit's system performance at a glance: https://status.modelbit.com/#system-metrics
• In order to automatically recover models with broken internal states, Modelbit now automatically restarts models that have many consecutive failures in a short time window.

🐛 Bug Fixes:

• Fixed a bug where deployments whose names are prefixes of other deployments' names could confuse the git sync.
• Made many improvements to GPU fallback logic and routing layer restart logic in response to: https://status.modelbit.com/incidents/xgsb85b1r8sh.

Week ending February 9, 2024

🛠️ Improvements:

• Added support for DataFrame inputs to mb.get_inference(...) when calling DataFrame-mode deployments!
• modelbit.get_inference(...) now handles chunking inputs over the wire and reassembling results so you don't have to!
• modelbit.get_inference(...) now preserves the indices in your input DataFrame when calling DataFrame-mode deployments.
• numpy.nan is now supported as a value in modelbit metrics (https://doc.modelbit.com/registry/metrics/)
• Added an ellipsis to truncated long inputs in the logs viewer so it's clear they were truncated.

🐛 Bug Fixes:

• Fixed a bug where non-numerical metrics could show up in the metrics graph selector.
• Fixed a bug where changing GPU settings after a failed environment build would cause the environment build logs to disappear.
• Fixed a bug where a deployment's logs could disappear from the cross-version logs viewer when a new version is deployed.
• Fixed a bug where mb.get_inference(...) did not work on branches with slashes in their names.
• Fixed a bug where async coroutines were hidden behind a zippy in the source code viewer.
• Fixed a bug that could cause the git sync to fail.
• Fixed a bug where the Modelbit-generated uncomment-to-run-locally code was wrong for DataFrame-mode deployments and async deployments.
• We upgraded all our runtimes this week in response to a new Linux container escape CVE. We were not impacted because we do not allow users to specify custom images, but we decided to apply the upgrade anyway out of an abundance of caution. There were no uptime impacts during the upgrade.

Week ending February 2, 2024

Model metric comparisons in the model registry! When models have the same metrics, you can visualize them with line charts and bar charts in the registry.

🛠️ More Improvements:

• Private packages are now default-excluded instead of default-included with new deployments. If their use is detected in notebooks, they are still automatically included in those notebook deployments. Learn more.
• Added truncation for very long log lines in the logs viewer.

🐛 Bug Fixes:

• Fixed a bug where the latest version of a private package is always included in notebook deployments instead of the specified version.
• Fixed issues with Modelbit Python compatibility with Python 3.7 and urllib3 < 1.26.0.
• Fixed a bug where non-Python server errors triggered an unrelated Snowflake error in the Snowflake API, making these errors unknowable from Snowflake.
• Fixed several bugs that led to query timeouts and errors when running single batches of hundreds of millions of inferences from Snowflake.
• Fixed a bug where going to "app.modelbit.com" would throw a server error if your last-viewed git branch had a slash in its name.
• Fixed a bug where linking to individual log lines would throw an error for Redshift and Snowflake log lines.
• Fixed various text wrapping bugs in the web application.
• Fixed a bug where the git sync status icon was sometimes overly optimistic about whether the git sync had really succeeded.

Week ending January 26, 2024

Modelbit is now SOC2 certified! To learn more, visit Modelbit's new Trust Center to download our SOC2 report and review other security measures that we maintain.

🛠️ More Improvements:

• Introducing the Python inference API! Get inferences from your deployed models right from your notebooks. Learn more!
• You can now optionally include or exclude common files from individual deployments! Learn more.
• You can now customize the max batch size sent from Snowflake to Modelbit for each deployment. This value gets passed directly to Snowflake as MAX_BATCH_ROWS. You can learn more here.
• Improved the performance of modelbit clone on large workspaces.

🐛 Bug Fixes:

• Made the error message more helpful for modelbit.add_package when the package already exists yet force=True was not specified.
• Fixed a couple intermittent crashes of Spark-NLP deployments.
• Fixed a bug where adding more than 100 models in a single call to the registry could fail.

Week ending January 19, 2024

Model metrics! You can now store arbitrary metrics on your model. Training jobs that save models to the registry will also show you those models' metrics. Now you can always know which models had the best precision, recall, and error scores. Learn more!

🛠️ More Improvements:

• You can now link to individual training job runs! Click out the little link icon next to the output of a job run to share that output with all your coworkers.
• Added support for async Python coroutines as inference functions in Modelbit.
• Improved the error messages and slack alerts of datasets that fail due to missing warehouses.
• Improved the readability of sklearn pipelines in model registry descriptions.
• Increased the max length of the name of a git branch created from Modelbit from 25 to 100 characters.

🐛 Bug Fixes:

• Fixed a bug where high-usage models could make it seem like other models weren't getting any usage at all.
• Fixed a bug where refreshing the model registry page and then clicking a model name led to a 404.

Week ending January 12, 2024

Integration with Tecton: Modelbit now integrates with Tecton! Any ML model and Tecton feature store can now work with your models deployed with Modelbit. Learn more.

🛠️ More Improvements:

• Common files are now implemented as symlinks in git rather than "by magic". In addition to making them more transparent, this will allow us to make several upcoming improvements to common files. Learn more.
• Added a status indicator in Settings to make it clearer when there are git sync problems.
• Made it clearer in individual deployments' usage charts when you're looking at usage for only one deployment version.
• Inactive branches are now hidden behind a link in Git settings. This should hopefully make the page more usable for those of you with lots of git branches!
• Improved Python environment detection for Spark-NLP deployments.
• Modelbit now takes you to your most recently used branch when you first log in.

🐛 Bug Fixes:

• Fixed a bug where the OpenAI integration failed to trigger for LangChain deployments.

Week of January 1, 2024

🎉 Happy new year from the Modelbit team!

Usage charts now show you average latency! Get model performance information at a glance.

🛠️ More Improvements:

• Modelbit now supports symlinks in deployment directories in git.

🐛 Bug Fixes:

• Fixed a bug where Python 3.6 deployments could fail to deploy.
• Fixed a bug where usage chart export data didn't exactly match the data in the chart visualization.

Week of December 25, 2023

Made various improvements to Modelbit's auto-scaling in bursty request scenarios.

🐛 Bug Fixes:

• Fixed a bug where branches appear inactive if they have inference requests but no recent pushes.
• Fixed a bug where long-running browser tabs open to very active logs viewers will just accumulate new log lines forever.
• Fixed a regression in our OpenAI integration that was caused by a change in OpenAI's API.

Week of December 18, 2023

Made it clearer on the Integrations page which integrations are active in your workspace!

🛠️ More Improvements:

• Made a number of improvements to optimize parallel inference handling.
• Improved the cold start time of deployments with large disk footprints.

🐛 Bug Fixes:

• Fixed a bug where long inference results were not properly highlighted in the logs viewer.
• Fixed a bug where inference functions in Redshift sometimes didn't update correctly.
• Fixed a bug where the "last deployed" time for a model sometimes got truncated.
• Fixed a bug where models sometimes didn't deploy to a non-main branch if main wasn't checked out before pushing to that branch.
• Fixed a bug where the rocket badge for GPU-enabled deployments sometimes didn't show up in the version dropdown.

Week of December 11, 2023

You can now link to individual inference log lines!

🛠️ More Improvements:

• Keep Warm is now available on non-main branches.
• Made it clearer when deployments have been archived on the deployment overview page.
• Improved the readability of errors in Hex notebooks.

🐛 Bug Fixes:

• Fixed a bug where deployments' REST API example calls didn't update very fast when "Require API Keys" was turned on.
• Fixed a bug where the "Redeploy as latest" button was disabled if there's only one version.
• Fixed a bug where deployments could drop pending async requests when they reboot.
• Fixed a bug where mb.add_models(...) could time out when adding thousands of models.

Week of December 4, 2023

Check out the new Model Overview page! See a useful summary of inference endpoints and training jobs in one concise view.

🛠️ More Improvements:

• The jobs logs viewer is now more useful and more visually pleasing!
• You can now run multiple training jobs in a single deployment simultaneously.
• Improved environment detection for Keras image models.
• Enabled "redeploy as latest" even when the version is already the latest!
• Lengthened the inference timeout for inference requests from Redshift.
• Improved the performance of the schema browser in the dataset editor for tables with hundreds of thousands of columns.

🐛 Bug Fixes:

• Cleaned up a couple confusing SQL error messages in datasets.
• Cleaned up some confusing warnings when deploying Keras models.
• Fixed a syntax bug in the auto-generated example Redshift query.

Week of November 27, 2023

Redesigned git settings: We made the git settings page easier to use and more visually pleasing.

🛠️ More Improvements:

• Added customizable inference timeouts with longer maximums! Learn more.

🐛 Bug Fixes:

• Fixed a bug where the loading animation for changes to training jobs was invisible.
• Fixed a spurious warning when using the newest version of Pandas with mb.get_dataset.
• Added a warning when using mb.deploy with old versions of the Modelbit package.
• Fixed a bug where git sync errors sometimes did not show up in the UI.
• Fixed a bug where Keras models sometimes failed to deploy from Windows environments.
• Fixed a bug where Modelbit sometimes put tensorflow-intel in environments deployed from Windows instead of tensorflow.
• Fixed a bug where calling Modelbit for inferences with strings-inside-strings caused errors.

Week of November 20, 2023

A10G GPUs with 24GB VRAM have arrived at Modelbit! You can change the compute environment of any version of any deployment after it is deployed. You can also set the compute environment using git or with the require_gpu parameter of mb.deploy. Learn more.

🛠️ More Improvements:

• Overhauled the UX of the Training Jobs page to make it more visually pleasing.
• Fixed various usability issues in training jobs and datasets.

Week of November 13, 2023

Improved the layout of the Deployments page, paying special attention to displaying long deployment names!

🛠️ More Improvements:

• Upgraded all our GPUs to run CUDA 12, as required by the latest releases of PyTorch.
• Improved environment detection for FastAI models.
• Made Markdown files look prettier in the source code viewer.
• Made it more intuitive to fix partially-completed warehouse connections.
• Improved the performance of large batches of inferences.
• Improved the performance of mb.get_dataset in high-concurrent-load scenarios.

Week of November 06, 2023

GPUs are now enabled for all customers! Toggle on GPUs in the UI for inference endpoints, or select a box with a GPU for training jobs, and feel the power course through you.

🛠️ Improvements:

• Model registry search is about 15x faster for customers with tens of thousands of models.
• Deployments that aren't changed or called for a while are now automatically archived.
• Archived models now correctly respect git branching and merging.
• Modelbit usage charts now break out CPU and GPU usage separately.

🐛 Bug Fixes:

• Fixed a bug where some users got 500 errors when logging in from notebooks.
• Fixed a bug where Modelbit overzealously auto-archived active deployments on new branches.
• Fixed a bug where Modelbit did not correctly detect the neptune[sklearn] package.

🎃 Week of October 30, 2023

🛠️ Improvements:

• Made the roles assumed by API keys clearer.
• Slack alerts are now available on all branches, not just main!
• Owners can now restrict Users from inviting new Users.
• modelbit.get_dataset() is now threadsafe.

🐛 Bug Fixes:

• Fixed a bug where training jobs created before their inference functions would never finish deploying.
• Fixed a bug where outputs from the Python logging library did not show up in the logs viewer.
• Fixed a bug where outputs to stderr did not show up in the logs viewer.
• Fixed a bug where Modelbit attempted to pickle modules instead of importing them if the module was assigned to a variable.
• Fixed a bug where newly-created branches would immediately show as inactive if branched from an inactive branch.
• Fixed a bug where llama_cpp deployments wouldn't include necessary system dependencies if deployed using git.
• Fixed a bug where we accidentally made most of our tooltips transparent.

Week of October 23, 2023

Merge deployments from within Modelbit! Now you can merge your deployments to main (or any other branch) without tabbing over to GitHub. This means we've also turned on branch protection even when you don't have a git integration set up.

Training jobs now show CPU and Memory Usage Statistics!

🛠️ More Improvements:

• Reordered training jobs logs previews to show you the most recent logs.
• Added a loading animation while training jobs are deploying.

🐛Bug Fixes:

• Fixed a bug with syntax highlighting of truncated responses in the logs viewer.
• Fixed a bug with spurious outputs from the add_job call when called from Hex.

Week of October 16, 2023

Deployments Sidebar Redesign: We reorganized the Deployments sidebar into sections! If you're looking for the Notebooks section, it's been folded into the Source Code section.

More Improvements:

• Modelbit now type-checks inputs to inference functions for Pydantic users!
• Modelbit's training jobs summary now includes which deployment version was used.
• Improved environment detection for Tensorflow models that use Keras.
• Improved the performance of the Usage Data graphs.

🐛 Bug Fixes: 

• Fixed a bug where if you use a custom PyTorch version locally, we always incorporated that version in production, even if you explicitly specified a different version in python_packages.
• Fixed various bugs related to non-Anaconda Windows environments.
• Fixed various search quality issues in the model registry search.
• Fixed usability issues relating to long model names and long source code file names.

Week of October 9, 2023

Slack Alerts: You can now get Slack alerts when datasets fail to update and when training jobs fail. Learn more.

More Improvements:

• Image files now render previews in the source code viewer.
• Added a loading bar when adding lots of models to the registry at once.
• Improved the performance of large GPU training jobs.
• Modelbit's logs viewer is faster.
• Improved environment detection for pycaret models.
• Improved the performance of models using PyTorch 2.1.0.
• Greatly improved the error messages from mb.get_model().
• Improved performance and memory usage when deploying models that contain extra_files.

🐛 Bug Fixes: 

• Fixed a bug where mb.models() did not work on branches with slashes in their names.
• Fixed a bug with syntax highlighting of truncated responses in the logs viewer.
• Fixed a bug where imports inside of functions were not detected as dependencies.

Week of October 2, 2023

Llama 2: You can now deploy Llama 2 Models in Modelbit with vastly improved support for llama.cpp. Learn more.

More Improvements:

• Improved the freshness and latency of Modelbit usage data.
• Added a helpful human-readable tooltip to the jobs cron scheduler.
• Made it easier to find the checkbox to require API keys.

🐛 Bug Fixes: 

• Fixed a bug where deployments would claim to be deployed sooner than they were.
• Fixed a bug where GPU training jobs sometimes did not report their usage.
• Fixed various UX issues caused by very long jobs log outputs.
• Fixed a bug where Modelbit's first-time user experience didn't work correctly in interactive Python terminals.
• Fixed a bug where Modelbit wouldn't connect to fresh git repos because they contain .git files.

Week of September 25, 2023

Training ML Models on GPUs: We've added support for training models on GPUs and on much larger boxes! Boxes even larger than the ones chooseable in the product are available upon request. Learn more.

Large Python Environments: We made it faster to build large Python environments as part of deploying large models! For example, the time to build a 6GB environment for OWL-ViT on JAX has been reduced from ~280s to ~180s. We are continuing to work on making it faster to build environments and deploy models.

More Improvements:

• We revamped the git connection wizard to make it easier to connect your own git repo.
• Modelbit is now available in AWS us-east-1! This is great news for Enterprise customers who want to leverage AWS PrivateLink in that region.
• Added support for Azure DevOps Git Repos. Learn more.
• We now support vertical pipes ("|") in model names in case you make questionable naming decisions.
• The mb.models() API call is now supported inside of deployments. Learn more.
• You can now add a secret header to Modelbit logs webhooks for security purposes. Learn more.

🐛 Bug Fixes: 

• Fixed a bug where training jobs that save changes on success would fail on protected branches.
• Fixed a bug where dataset refresh histories were displayed in UTC instead of your local timezone.
• Fixed a bug where models in the model registry sometimes did not branch correctly.
• Fixed a bug where scheduled training jobs sometimes appeared to time travel into the past when running.

Week of September 18, 2023

Improvements:

• Improved support for AWS PrivateLink so that inference network traffic doesn't have to travel across the public internet! Learn more.
• We made the ~/.cache directory writeable as some model frameworks write temporary files there.
• Fixed a bug where models in the registry were not correctly merged to main during a branch merge.

Week of September 11, 2023

Async API: For long-running inferences, you can now supply a webhook for Modelbit to call you back when your results are ready. Learn more.

More Improvements:

• Modelbit's source code viewer now deeplinks to the models you load from pickles or the registry.
• Environments including the "jax" package now automatically include the version with the right cuda drivers.
• Modelbit now supports returning instances of user-defined classes from inference functions.
• Improved the error message when trying to use the old Modelbit package version that comes bundled with Hex.

🐛 Bug Fixes:

• Fixed a bug where a Modelbit warning message suggested the wrong package name for the "cv2" module.
• Fixed a bug where Modelbit's auto-generated git commit message was wrong for some changes to endpoints.
• Fixed a bug where the "extra_files" parameter on “modelbit.deploy” sometimes silently failed.
• Fixed a bug where Modelbit's dynamic notebook outputs were not always rendering in Hex notebooks.

Week of September 4, 2023

Endpoints Improvement: Modelbit endpoints can now point to deployments on any git branch. This should make it seamless to ship endpoint changes from a branch using a merge request. Learn more.

Model Registry Improvements:

• The Python dependencies of any models used from the model registry in your inference function in a notebook will automatically be added to the deployment's Python environment once it's deployed.
• If you use the model registry, Modelbit will automatically log what model was pulled from the registry at inference time.
• We improved the search performance of the model registry.

More Improvements and 🐛 Bug Fixes:

• Modelbit now shows the Python package dependencies of pickled models in the source code viewer.
• It’s much faster to deploy models that depend on old versions of PyTorch.
• We released an integration with Neptune.ai!
• We greatly improved Modelbit’s integration with DataDog.
• We made various performance improvements and bug fixes to Modelbit’s Snowflake API.

Week of August 28, 2023

🎉 Announcing the Modelbit Model Registry!

• You can now add models to Modelbit and use them in running deployments, all without redeploying your endpoints.
• The model registry enables you to manage models separately from deployments. This allows you to version the model separately from the code of the deployment, to use the same model in multiple deployments, or to load different models dynamically based on runtime parameters.
• Learn More!

More Improvements:

• You can now see the last time that API Keys were used.
• We made performance improvements to Modelbit’s logs viewer.
• We improved the the branch picker to show the most recently used branches.
• We’ve significantly sped up large responses from Modelbit.

🐛 Bug Fixes:

• We fixed a bug when adding a new warehouse to Modelbit.

Week of August 21, 2023

GPU Cold Start Times: Your models running on GPUs will now have much faster cold start times.

CORS Headers: We added support for CORS Headers so you can call Modelbit directly from a JS Client.

More Improvements:

• Modelbit will now automatically detect your Python packages installed from Git and install them that way in production.
• Improved PyTorch environment build times.
• Improved environment detection for Llama models.
• Improved environment detection for Tensorflow-Recommenders models.
• We improved the performance of "modelbit.log_image".

🐛 Bug Fixes:

• Fixed a bug where very large log batches didn’t show up in the log viewer.
• We fixed a bug where “functools.lru_cache” was not running correctly in Modelbit.

Week of August 14, 2023

Common Files: Files that you commonly use in model deployments can be checked into Modelbit as common files. Common files are automatically added to all deployment environments.

More Improvements:

• We improved how we render Markdown files in Modelbit’s source code viewer.
• We improved Modelbit's support for LayoutLMv3 models.
• We added a search bar to our docs page.

🐛 Bug Fixes:

• We fixed an environment detection issue for Segment-Anything models running on GPUs.
• We fixed a bug where you couldn’t switch to using GPUs on non-main branches.

Week of August 7, 2023

PyTorch environment detection: We’ve made a number of improvements to environment auto-detection for PyTorch models.

• Modelbit will now detect Torch and Torchvision in Segment-Anything models.
• We fixed an issue where Modelbit would throw erroneous warnings for PyTorch.

Runtime Context Commands: Your code can now learn about its runtime context when running in Modelbit! In particular:

• mb.in_modelbit() tells you whether your code is running in Modelbit.
• mb.get_branch()  tells you what branch your code is running on.
• mb.get_deployment_info()  will additionally tell you the name and version of the deployment that's running.

More Improvements:

• Increased performance for Tensorflow and PyTorch models.
• Improved the file that is automatically created when you connect Modelbit to your GitHub/GitLab by including more helpful links to relevant docs pages.
• Fixed an issue with workspace auto-naming.

Week of July 31, 2023

Role based access controls: You can now control which users can deploy models, change settings, and update billing information with role based access controls (RBAC) to make it easier for you to manage permissions for existing users and new users.

More Improvements:

• Reduced Cold Start Time for Pytorch models.
• We made various performance improvements to modelbit.get_dataset in deployments.
• Fixed multiple bugs for Python environment detection in notebooks (Especially for pytorch-ie models).

Week of July 24, 2023

Improved Deployment Card Layout: When you are working on a branch, the deployments you’re working on are promoted to the top of the list of deployment cards for quicker access.

Deployment Logs Webhook: We’ve added a deployment logs webhook so that you can send Modelbit logs in real time to a variety of logs ingestion services. This allows you to do things like ingest Modelbit logs into a database for custom reporting.

Arize Integration: We’re excited to announce that we’ve built an integration with Arize! This new integration will support detailed inference monitoring and analysis for models running in Modelbit.

More Improvements:

• We've made it easier to join existing Modelbit workspaces when signing up with your work email address.
• We’ve released a new feature that shows users environment build time predictions. With this new feature, you will know how long it’ll take to build your next Torch or XGBoost environment.
• We’ve reduced the refresh time of Snowflake and Redshift datasets by about 40%.

Week of July 17, 2023

Logging images to Modelbit: You can now log images in addition to text in Modelbit! Use log_image to log matplotlib Figures or image files to Modelbit. Those images will be rendered directly in the logs viewer.

More Improvements:

• You can now set users as a Billing Admin when you invite them instead of having to wait for them to accept their invitation.
• Datasets now have an inline refresh button so you don't have to edit the dataset to refresh it.
• When deploying a model that depends on torch, Modelbit now automatically includes the pytorch.org repository so you can deploy PyTorch models with custom CUDA dependencies.
• Calling mb.deploy from a notebook no longer pickles basic primitives, which makes code reviews easier to read in your merge requests.

Week of July 10, 2023

Protected branches for git workflows: Teams using Modelbit can now protect certain branches to ensure all changes come from approved merge requests. Learn More.

Weights & Biases Integration: We've added this new integration to enable training in Modelbit and experiment analysis with Weights & Biases. Learn More.

More Improvements:

• Notebook users can now download files created while running training jobs. Modelbit will also show the output incrementally during long runs.
• Added support for deployments to return very large REST responses with the “links” response format.
• git push will now show much clearer file format errors.
• We’ve reduced environment build times by up to 65%. Deployments with new dependencies will become available much quicker!

Week of July 3, 2023

Support for AWS Athena: In addition to Snowflake and Redshift, Modelbit customers can now make datasets and feature stores by querying Athena warehouses.

REST API Performance: We’ve improved the performance of inferences using the REST API with Keep Warm by reducing network latency up to 50%.

More Improvements:

• Modelbit's Training Jobs UI now includes a button to stop the execution of any training job that was started unnecessarily.
• We improved the notebook login flow by adding support for switching to a specific branch while authenticating to Modelbit.
• Modelbit now includes auto-detected packages in the Python environment, even when you specify some of your packages.
• Notebook deployment workflows in Modelbit now include helpful warnings for dependencies missing from extra_files, misspelled packages in python_packages, and deployments that should be using dataframe_mode=True.
• Deployment logs in Modelbit now show more helpful errors if deployments experience certain errors from Pydantic validators.
• Fixed an issue where specifying certain versions of boto3 in the requirements of a private package could prevent the deployment’s environment from building.

Week of June 26, 2023

More sophisticated drift charts: In models that return JSON (i.e. most of them) Modelbit will plot the results from every JSON key on the drift chart.

More Improvements:

• Datasets with multi-million row feature stores are much faster.
• Log viewer will show which API keys were used.
• Modelbit now detects dependencies of source files specific with extra_files.
• Modelbit datasets will now always recognize and respect SQL data types.
• Fixed issue with network timeouts causing exceptions in some API calls from the Modelbit Python package.

Week of June 19, 2023

Training Jobs 2.0! The API and web UI for training jobs got a major overhaul! It's much easier to kick off an arbitrary job from a notebook even without an associated deployment. The API for jobs got a whole lot simpler. You can fetch the results of any job run from a notebook, and you can manually look at results and then decide whether to redeploy your deployment with the results of that job run! Learn more.

More improvements:‍

• Error messages from add_files got a lot clearer.
• Modelbit output is easier to read in Hex's dark mode.
• Custom python_packages syntax now supports special dependency syntax, e.g. "torch==2.0.1+cu118"

Week of June 12, 2023

Improved logs viewer: Standard out and standard error are now separated for easy scanning. Plus, the logs viewer now supports way more than 5,000 characters per log line.

More improvements:‍

• Fixes to the first-time user experience for users who log in with an emailed token.
• Errors due to large REST payloads are now clear!
• Fixed a bug specifying private packages that have hyphens in their names.

Week of June 5, 2023

Automatic system package detection: When Python packages depend on Linux system packages, those system packages are now automatically included in runtime environments as necessary. For example, all Python environments that include torch now also include build-essential.

Week of May 26, 2023

Deploy with extra files: Bring extra local files to production alongside your deployment with the extra_files parameter on modelbit.deploy. Learn more.

Week of May 15, 2023

Secrets Management: Secure storage and management for your AWS Secret Access Keys, OpenAI API Keys, and any other secrets to make them securely available at inference time. Learn more.

Week of May 8, 2023

Automatic notebook environment detection: When deploying from a notebook, Modelbit now automatically detects which Python packages your code depends on, and the versions of those packages, and makes sure they're installed correctly in production. You should no longer need to specify Python packages at deploy time in most cases.

Dependences from remote git repos: Include Python packages in your deployment that are installed from remote git servers, like detectron2, segment-anything, and pretty much everything from Facebook Research! Specify them just like you would in a requirements.txt file.

Week of May 1, 2023

Brand new infrastructure for big neural nets! Your Tensorflow models, PyTorch models, big fancy transformers, and other deep neural nets now run on infrastructure dedicated to their performance! Modelbit will detect models with specific memory and compute needs, including deep neural nets, and deploy them to specific infrastructure optimized for their stability and performance.

Week of April 24, 2023

Deploy with Private Packages and Modules: Add your internal helper and utility packages to Modelbit so they'll be available for all of your deployments in production! Run modelbit.add_module to add an imported module or modelbit.add_package to add a Python package. Learn more.

Week of April 17, 2023

Archive your deployments when you're no longer actively working on them so you can keep a clean workspace, especially on your development branch!

Week of March 20, 2023

Slack Alerts: Get alerted whenever your model misbehaves! Modelbit now integrates with Slack so you can get alerted any time a model throws an exception for any reason. Learn more.

Singleton Mode is an easier API for getting a single inference rather than inferences in batches.

Logs are much easier to read for complex inputs, outputs, and log lines.

Week of March 13, 2023

Endpoints: A/B test your models, send traffic to multiple models at once, canary test your models and more! Learn more.

Eppo Integration: Integrate with Eppo to take advantage of even more sophisticated A/B testing functionality. Learn more.

Week of March 6, 2023

Variable Size Job Runners: Training jobs can now be run on a variety of different CPU and memory configurations, for when you need to burst up to 120GB RAM to train your models. Learn more.

Support for Python 3.11! 🐍

Week ending April 19, 2024