> For the complete documentation index, see [llms.txt](https://timberid.gitbook.io/timberid/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://timberid.gitbook.io/timberid/architecture-of-timberid/detailed-design/research-colabs/model-training-variational-inference.md).

# Model Training (Variational Inference)

Last Updated: 2023-09-27

## Background

TimberID includes many techniques for generating [isoscapes](https://en.wikipedia.org/wiki/Isoscape) from geospatial point-measurements, ranging from Kriging interpolation to our most best-performing variational inference deep learning model. Of all the alternatives we evaluated, our best results come from predicting the isotope distribution with variational inference and a deep neural network, which appears to be a novel technique for isoscape generation. We focus here on producing isoscapes with variational inference, but the same general principles apply to most of our isoscape models.

## Brief Guide: Model Training

Find the Colab at: <https://colab.research.google.com/github/tnc-br/ddf-isoscapes/blob/main/dnn/variational_inference.ipynb>

Most users should obtain somewhat reasonable predictions out-of-the-box by running the training notebook with the default configuration options after running the [data ingestion notebook](/timberid/architecture-of-timberid/detailed-design/research-colabs/data-ingestion.md) with its default options and export path. However, our machine learning models are still one-off artifacts since we are rapidly iterating on model research. The TimberID processes for producing production-quality models are not automatic. For now, this notebook only attempts to provide a reasonable baseline as a place to start for running model experiments.

A few important, high-level notes:

* Some overfitting is normal, but training loss should not *significantly* dip below validation loss. Example of a typical loss curve:\
  ![](https://4155068849-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FkD1zcJTeGwRpwjArj78G%2Fuploads%2F1UE3InwIBf01FrUzloBS%2Fimage.png?alt=media\&token=cc40733b-ff76-4ed8-a117-b384adabf2a5)\
  \
  If `loss` drops significantly below `val_loss`, the model has most likely memorized its training set and is unlikely to produce a good isoscape. Common reasons for this are: not enough data, a training distribution that is not diverse enough, training for too many epohcs (overtraining). To mitigate overtraining, the notebook comes with early stoping and a default `patience` setting of 30 epochs.
* For reproducible results, be sure to re-run the cell that contains `tf.keras.utils.set_random_seed(18731)` before each training run. This model is very small, which makes it highly sensitive to initialization.
* We use Xavier initialization but have not performed robust experiments to compare alternatives other than no initialization at all.
* The code under "Run this cell to generate an isoscape" does not appear to be fully functional at this time. For generating isoscapes, prefer the [Isoscape Generation](/timberid/architecture-of-timberid/detailed-design/research-colabs/isoscape-generation.md) notebook instead.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://timberid.gitbook.io/timberid/architecture-of-timberid/detailed-design/research-colabs/model-training-variational-inference.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.