1
$\begingroup$

Start with $N>3$ vectors $\vec{v}_I$ in $\mathbb{R}^3_+$, any $3$ of which are linearly independent. $I$ here ranges from $0$ to $N-1$.
Let $v_{\left[abc\right]}$ be a matrix in $\mathbb{R}^{3 \times 3}_+$ that picks three of the (column) vectors $\vec{v}_a\, \vec{v}_b\, \vec{v}_c$ in order $a < b < c$, without duplication. There are $\frac{1}{3!}N \left(N-1\right) \left(N-2\right)$ such matrices, and $\left[abc\right]$ enumerates them.

For each $v_{\left[abc\right]}$, find the linear combination $\vec{w}_{\left[abc\right]}$ that lands on a particular arbitrarily chosen point $\vec{p}$, i.e. $v_{\left[abc\right]} \vec{w}_{\left[abc\right]} = \vec{p}$.
Discard those linear combinations, which contain negative coordinates. Of the remaining, now non-negative combinations, if there are any, find the average. (Otherwise, return nothing)

$$\begin{align*} {w_{\left[abc\right]}}_i &= \sum_j {{v}_{\left[abc\right]}^{-1}}_{ij} p_j \\ \omega_{\left[abc\right]} &= \begin{cases} 1 & \text{if } 0 \le \min_i\left({w_{\left[abc\right]}}_i\right) \\ 0 & \text{else} \end{cases} \\ n &= \sum_{\left[abc\right]}\omega_{\left[abc\right]} \\ w_i &= \frac{1}{n} \sum_{\left[abc\right]} {w_{\left[abc\right]}}_i \omega_{\left[abc\right]} \text{ if } n>0 \end{align*}$$

An alternative filtering method also filters out linear combinations greater than some constant, say, $1$:

$$ \omega_{\left[abc\right]}= \begin{cases} 1 & \text{if } 0 \le \min_i\left({w_{\left[abc\right]}}_i\right) \le 1 \\ 0 & \text{else} \end{cases} $$

and instead of the unweighted average, a sample from an $n$-choice dirichlet distribution could also be taken as a weighted average, to sample from the space of all weight combinations that satisfy the filtering constraint. (I hope this is a correct statement. I haven't actually formally proved this part)

Now the question is, is there a method to calculate this, that is more efficient, than explicitly following this recipe? The $\frac{1}{3!}N \left(N-1\right) \left(N-2\right)$ very quickly makes this intractable for large $N$. In my use case, I'm looking for an $N$ somewhere between $400$ and $500$, giving me, in the worst case, $0 \le \left[abc\right] < 20708500$, i.e. over 20 million matrix inversions for evaluating the correct weights for a single point.

Note: Constrained Optimization methods can absolutely find one solution that fulfills the constraints, but this method, if I did it right, finds, and can sample from the space of all of them. This is a desirable property for me. If the ability to sample from the full space makes this difficult, just finding precisely the (unweighted) average solution in a less resource-hungry way would also be appreciated.


I don't need absolutely insane speeds. It just needs to be tractable. I plan to implement this in python/numpy, so I don't expect ultra-optimized peak performance. But any way to save time and memory is good. Particularly great would be a "local" version of this, that lets me directly calculate a particular $w_i$ with minimal effort.


I hope the notation here is clear enough. Some of the dimensionalities here are tricky so I'm not 100% sure that I got it right. Please ask if anything doesn't quite make sense.

$\endgroup$
6
  • 1
    $\begingroup$ Correct me if I'm wrong, but is another way of thinking about it that you want to randomly sample points inside a convex polytope? $\endgroup$
    – Pseudonym
    Commented Nov 1, 2022 at 23:02
  • $\begingroup$ Please describe how you form the matrix $v_{\left[abc\right]}$ from the three vectors $\vec{v}_a\, \vec{v}_b\, \vec{v}_c$. Are those the rows of the matrix, or the columns of the matrix? I'm guessing they are the columns? $\endgroup$
    – D.W.
    Commented Nov 1, 2022 at 23:49
  • $\begingroup$ 20-million matrix inversions of 3x3 matrices sounds likely to be pretty fast to me. I bet it can be done in under a second if implemented in C using floating point arithmetic (assuming you're not too worried about numerical precision/roundoff/overflow/underflow errors with using floating point arithmetic). Have you tried implementing it? How fast is it? How fast do you need it to be? $\endgroup$
    – D.W.
    Commented Nov 2, 2022 at 0:03
  • $\begingroup$ @D.W. yeah they would be the columns $\endgroup$
    – kram1032
    Commented Nov 2, 2022 at 10:02
  • $\begingroup$ @Pseudonym yeah this should be equivalent to sampling points inside a polytope I think, although I'm not sure it's technically necessarily convex $\endgroup$
    – kram1032
    Commented Nov 2, 2022 at 10:03

1 Answer 1

1
$\begingroup$

I suggest you randomly sample 10,000 matrices $v_{[abc]}$. Keep only the ones where the resulting linear combination $w_{\left[abc\right]}$ meets your filtering constraints (e.g., is all positive). Average the ones kept. Output that as an estimate of the value you are seeking.

This should be a pretty good estimator of the value you are computing. You are trying to compute the expected value, with the expectation taken over all $N(N-1)(N-2)/6$ combinations, and this is a finite-sample estimate of that expectation. To put it another way, the sample mean is a good estimator for the population mean, and this method computes the sample mean. We're essentially using rejection sampling here to address the filtering condition.

How good will this be? That will depend on how many matrices/combinations you keep, with this method. Heuristically, I expect that the linear combination has about a probability $1/2$ to be non-negative in each coordinate, or about a $1/8$ probability to be non-negative in all three coordinates. So if your filtering condition is "all three coordinates must be non-negative", then I expect that you'll keep about $10000/8 \approx 1250$ combinations, which should be enough to get a good estimate of mean.

In comparison, if you had a filtering condition that was satisfied by only a miniscule fraction of matrices, then this method would not be a good choice.

$\endgroup$
7
  • $\begingroup$ I'll try this in a bit, and if it works well, I'll accept this, thanks :) $\endgroup$
    – kram1032
    Commented Nov 2, 2022 at 10:05
  • $\begingroup$ There is a chance, that I sample a point from quite close to, or even exactly a particular vector, which would result in most situations violating my constraints, meaning acceptance rate would be quite low. Any way around that, or would I have to just bite the bullet and explicitly evaluate all in that case $\endgroup$
    – kram1032
    Commented Nov 2, 2022 at 10:14
  • $\begingroup$ @kram1032, why would the acceptance rate be very low if $p$ is near $a$ (say)? For instance, acceptance is 100% if $p=a$. $\endgroup$
    – D.W.
    Commented Nov 2, 2022 at 12:34
  • $\begingroup$ Say I have 500 vectors. p happens to be colinear with the first vector (but I don't know that). Then I think get 124251 matrices in total where I get a non-zero contribution out of 20708500 - so just 0.6% of the matrices. If I randomly sample them like this, that would be my acceptance chance, right? (the remaining matrices do not contain this one vector, so none of them will give a contribution) $\endgroup$
    – kram1032
    Commented Nov 2, 2022 at 12:52
  • 1
    $\begingroup$ @kram1032, my advice would be to try the simple thing, see if it works, and worry about this case only if you experience problems (e.g., if the filtering filters out a lot more than 90% of the samples). But I imagine you've thought about this more than I have, so you will probably know best what is best. $\endgroup$
    – D.W.
    Commented Nov 4, 2022 at 1:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.