# How to correlate a matrix of values to get a coordinated point?

I got a n*m matrix updated in realtime (i.e. about every 10ms) with values between 0 and 1024, and I want to work out from that matrix a multitouch trackpad behaviour, which is:

• generate one or more points on the surface given the values on the matrix,
• make this or those point as big as the value can be.

For example here is a few lines of a 9x9 matrix updates, and we can consider the following matrix as an example (with a touch in the middle):

[ [ 12,  7,12 ],
[ 12,129,19 ],
[ 12, 11,22 ] ]


The goal is to mimic the behaviour of a common touchpad (like on every smartphone, or laptop). So, I'm getting values from a evenly distributed matrix of capacitive sensors on a physical object, which are processed by a microcontroller into a matrix, and I want to get coordinates and weight of one or several points.

The idea would be to get something like this (of course, I don't expect to have more than 2 or 3 detected points, and that level of precision with a matrix that small).

Here are a few example raw logs:

Edits:

Thinking about my problematic made me consider this idea: I think I should make some kind of interpolation to augment the definition of the matrix, and in some way make the new values additive.

i.e. imagine we have the following matrix :

[ [ 200, 200, 150 ],
[ 150, 150,  80 ],
[  80,  80,  40 ] ]


and we want to interpolate it somehow into something that would look like (I'm inventing the values, but it's to expose the idea):

[ [ 200, 400, 200, 175, 150 ],
[ 175, 200, 175, 150, 125 ],
[ 150, 170, 150, 125,  80 ],
[ 100, 125, 100,  80,  60 ],
[  80,  80,  80,  60,  40 ] ]


I've looked at interpolation algorithms, and it looks like the one we want that is the closer to our needs is the hermite interpolation. But though I have RTFM on interpolation methods, I don't know how I can apply it to a matrix.

• Welcome! It's hard to tell what you are asking. Please specify more precisely what the input and output of the desired algorithm are. – Raphael Oct 29 '12 at 21:55
• I will record and share recording of the matrix for different simple gestures (like one touch traversing the matrix from top left to bottom right, and two touches from top to bottom on left and right). – zmo Oct 30 '12 at 8:31
• So the basic assumption is that the values are high around the position where the pad is touched, low where it is not, and you want to get (approximate) coordinates of touches based on those readings? – Raphael Oct 30 '12 at 10:22
• Indeed, that's what I want. I'd like to get a weight also (to get how big a touch is), but that's secondary. At least, if I can get directions about that, I'd be really glad. – zmo Oct 30 '12 at 11:07
• I think we can assume that a touch is defined by a value over a "trigger" limit constant which could be something like 100. But because of the properties of a capacitive sensor, all values around it are influenced by the touch. – zmo Oct 30 '12 at 11:57

## 1 Answer

I finally managed to get what I want. I used python's scipy's

interpolate.RectBivariateSpline(x,y,z,kx,ky)


to create a 60x60 matrix out of the 3x3 matrix. And out of that, I used opencv libraries to detect blobs (mainly, findContours() and fitEllipse()), so now I got a list of ellipses matching the several touches I have.