Alex Shtoff
Intro Up until now in the series we discussed mainly the expressive power of different variants of our model family, \[f({\boldsymbol x};{\boldsymbol A}_{0..n}) = \lambda_k \Bigl({\boldsymbol A}_0 + \sum_{i=1}^n x_i {\boldsymbol A}_i\Bigr),\] where each \(\boldsymbol A_i\) is a symmetric...
Alex Shtoff
Intro In the last post we discussed the meaning of our model family \[f({\boldsymbol x};{\boldsymbol A}_{0..n}) = \lambda_k \Bigl({\boldsymbol A}_0 + \sum_{i=1}^n x_i {\boldsymbol A}_i\Bigr),\] where each \(\boldsymbol A_i\) is a symmetric matrix. In the last post we discussed what these models...
Alex Shtoff
Intro We continue our discussion of machine-learned models of the form \[f({\boldsymbol x};{\boldsymbol A}_{0:n}) = \lambda_k \Bigl({\boldsymbol A}_0 + \sum_{i=1}^n x_i {\boldsymbol A}_i\Bigr),\] where \({\boldsymbol A}_i\) are learned symmetric matrices, and \(\lambda_k\) is the \(k\)-th...
Alex Shtoff
Intro Efficiency is a quite superpower in research. When training is fast, you can iterate: try an idea, get surprised, debug, tune, and move on. When training is slow, you start avoiding experiments you should be running, simply because they cost too much time. This became very tangible in this...
Alex Shtoff
Intro When trying to model a complicated relationship between features, our go-to architectures are typically either neural networks or decision trees. They are well-established, well-studied, and have an abundance of software for training them. So why not? But sometimes we have some additional...