From Kriging to Machine Learning: how estimation methods find their weights
Kriging, radial basis functions, support vector machines and neural networks are different ways of finding the weights for a weighted-average estimate.
Kriging finds weights from a variogram model of how similar samples are at different distances. Radial basis functions (RBFs) and support vector machines (SVMs) use the same idea, with a decaying function called a kernel, but solve for the weights once and reuse them, so later estimates are much faster. Neural networks reach a similar result by training. The method changes speed and behaviour, not the basic principle. This is part 2 of 4.
Kriging: a better way to find the weights
Part 1 showed that an estimate depends only on its weights. Kriging is a way of finding the best weights from the sample locations themselves. For a deposit, the samples are grade values at X, Y, Z locations, and the estimate is made at locations where nothing was sampled, for example the cells of a grid or the points of a grade shell.
In ordinary kriging, finding the weights is a two-stage process. First, the similarity between every sample and every other sample is calculated with a variogram model. Second, the similarity between the location being estimated and each sample is calculated with the same model. The two are combined to give the final weights.
The variogram
Just as IDW lets you choose how quickly influence drops with distance (distance, squared, cubed), a variogram model, usually spherical, Gaussian, exponential or a combination, sets how quickly a sample loses influence. Variography is the process of fitting that model to your own data.
From kriging to implicit modelling with RBFs
A variogram model behaves very much like a radial basis function (RBF): a function that decays as distance grows. That is why implicit modelling uses RBFs, most often Gaussian or spheroidal ones. The function is, once again, only a way to adjust the weights with distance.
Why RBFs are faster
In ordinary kriging, both stages are repeated for every estimate, which becomes very expensive for large datasets. An RBF interpolant works the other way round: the weights between each sample and every other sample are found once, in a step called fitting, and reused for every estimate afterwards. Fitting takes some time, but evaluation is then fast.
Samples can also be included selectively. Kriging usually does this with a search radius. Fast RBF methods use the distances between samples to skip points too far away to matter, which is comparable to a search ellipsoid.
From RBFs to machine learning
Support vector machines (SVMs) fit weights in a way very similar to RBFs, with slightly different mathematics. SVMs call the decaying function a kernel, which is just another name for an RBF or variogram model. Their benefit is that they keep only the most important samples and ignore the rest, as in the Dutch-baby example in part 1.
Neural networks do the same job in another way. Each node holds an activation function (often Gaussian or sigmoid) and a weight, and their outputs are multiplied and added, as in a weighted average. The network is trained by estimating a known sample, measuring the error and adjusting its weights, which is effectively trial and error.
IDW, kriging, RBFs, SVMs and neural networks all produce weighted averages. What differs is the function that decays with distance, the range of influence, and how the weights are found.
Apply it
Choose the estimator that fits your data
In GEOREKA, IDW, RBF and SVR are available in Core, with dual ordinary kriging in Premium, so you can compare methods on the same samples. Resource estimation in GEOREKA →
Common questions
From kriging to machine learning FAQ
What is the difference between kriging and RBF interpolation?
Both compute weights from a function that decays with distance. Ordinary kriging recalculates the weights for every estimate, while an RBF interpolant fits the weights once and reuses them, which makes evaluation much faster.
Is a kernel the same as a variogram model?
They play the same role, setting how quickly the influence of a sample falls with distance. Different methods use different names: variogram model, radial basis function, kernel or activation function.
Why are RBFs faster than kriging?
Because the weights between samples are found once, during fitting, and then reused for every new estimate, instead of being recalculated each time.