How do I fit a common parameter between two sets of equations?
January 8, 2014 2:49 PM Subscribe
I have two observables that I'm measuring, x and y. Let's say both are functions of time t: x = f(t) and y = g(t). Given several measurements [(x,y) at (t1, t2, t3...)], I want to fit for the parameters of f() and g(). What's the right way to do this when f() and g() have a parameter in common?
For concreteness, suppose:
x = a + b*t + e*sin(t);
y = c + d*t + e*cos(t).
If they didn't have e in common, I'd just fit the two sets of equations independently. But how do I do this with the linkage?
(I vaguely remember connecting the two equations with (x-f(t)) + lambda*(y-g(t)) = 0 but I don't remember where to look up the details...?)
For concreteness, suppose:
x = a + b*t + e*sin(t);
y = c + d*t + e*cos(t).
If they didn't have e in common, I'd just fit the two sets of equations independently. But how do I do this with the linkage?
(I vaguely remember connecting the two equations with (x-f(t)) + lambda*(y-g(t)) = 0 but I don't remember where to look up the details...?)
Why not just define a new variable z = x + y (and accordingly, z(t;...) = f(t;...) + g(t;...)) and then regress on that? In that case, z will be a function of all your parameters of interest and the best estimates for the parameters of z will also be the best estimates for those of f and g. This all also depends on how you're going about the fitting/regression; this link might be of interest, even if you're not using R for this.
posted by un petit cadeau at 3:35 PM on January 8, 2014
posted by un petit cadeau at 3:35 PM on January 8, 2014
I would use a dummy variable to define "x or y status", and then regress on a new variable z:
z = D1*[a + b*t + e*sin(t)] + D2*[c + d*t + e*cos(t)],
where D1=1 where z=x and 0 otherwise. Similarly, D2=1 where z=y and 0 otherwise. Fit for a,b,c,d, and e however you were going to fit f(t) and g(t) without the common e.
(Also, it goes without saying, but the best answer to this question is going to vary based on your application.)
posted by tinymegalo at 4:26 PM on January 8, 2014
z = D1*[a + b*t + e*sin(t)] + D2*[c + d*t + e*cos(t)],
where D1=1 where z=x and 0 otherwise. Similarly, D2=1 where z=y and 0 otherwise. Fit for a,b,c,d, and e however you were going to fit f(t) and g(t) without the common e.
(Also, it goes without saying, but the best answer to this question is going to vary based on your application.)
posted by tinymegalo at 4:26 PM on January 8, 2014
Best answer: This is still a linear model in the parameters, so you can still use standard least-squares techniques like you would with a simpler model. Specifically, denote your measurements at t_1, t_2, ... as f_1, f_2, ... and g_1, g_2, ... You can still construct the least-squares quantity
L = ∑ (f_i - f(t_i))2 + (g_i - g(t_i))2
summed over i. This will be a quadratic function in a, b, c, d, and e, and so will have a unique minimum. The values of the parameters a–e that minimize the quantity L are the parameter values with maximum likelihood.
This assumes, of course, that the measurements are uncorrelated and have normal error distributions with the same standard deviation. It's not too hard to relax a few of these restrictions if you need to, though. (Except for the normal error distribution on the measurements.)
posted by Johnny Assay at 4:50 PM on January 8, 2014 [1 favorite]
L = ∑ (f_i - f(t_i))2 + (g_i - g(t_i))2
summed over i. This will be a quadratic function in a, b, c, d, and e, and so will have a unique minimum. The values of the parameters a–e that minimize the quantity L are the parameter values with maximum likelihood.
This assumes, of course, that the measurements are uncorrelated and have normal error distributions with the same standard deviation. It's not too hard to relax a few of these restrictions if you need to, though. (Except for the normal error distribution on the measurements.)
posted by Johnny Assay at 4:50 PM on January 8, 2014 [1 favorite]
Best answer: This isn't fundamentally different from the case where you only have one observable, or where you have two observables without any parameters in common.
How are you "fitting" the data? If you are minimizing the least squares difference between the fitted points and the observed points, then the difference between the observed point (x(t), y(t)) and the fitted point (f(t), g(t)) is [x(t) - f(t)] ^ 2 + [y(t) - g(t)] ^ 2, and solving for the parameters really isn't any different from the simpler cases -- you just have a parameter that shows up in two different places in the equation. Likewise if you fit the data by using maximum likelihood.
Your comment about the lambda approach sounds like the method of Lagrange multipliers, which may be a way of fitting f(t) to x(t) subject to the constraint that g(t) = y(t) but I don't think that's what you want in this case.
posted by leopard at 6:34 PM on January 8, 2014
How are you "fitting" the data? If you are minimizing the least squares difference between the fitted points and the observed points, then the difference between the observed point (x(t), y(t)) and the fitted point (f(t), g(t)) is [x(t) - f(t)] ^ 2 + [y(t) - g(t)] ^ 2, and solving for the parameters really isn't any different from the simpler cases -- you just have a parameter that shows up in two different places in the equation. Likewise if you fit the data by using maximum likelihood.
Your comment about the lambda approach sounds like the method of Lagrange multipliers, which may be a way of fitting f(t) to x(t) subject to the constraint that g(t) = y(t) but I don't think that's what you want in this case.
posted by leopard at 6:34 PM on January 8, 2014
Response by poster: Yeah I think you guys are right, I'm overthinking this. I do want the least squares difference minimized, and minimizing ∑{(xi-f(ti))^2 + (yi-g(ti))^2} should be sufficient.
Now to see if I can plug this straight into a fitting routine in SciPy...
posted by RedOrGreen at 8:12 PM on January 8, 2014
Now to see if I can plug this straight into a fitting routine in SciPy...
posted by RedOrGreen at 8:12 PM on January 8, 2014
This thread is closed to new comments.
posted by supercres at 3:32 PM on January 8, 2014 [1 favorite]