Alright, let’s start with a problem that you might want to solve if you were driving a “space shuttle” in the middle of deep space (perhaps in a video game). You have 5 orbs floating in space at the same locations of the 5 points picked in the “Building a Curved Path” (cubic splines) post a few weeks ago.

The problem is: You want to fly through the same 5 points and catch the orbs in your net as you pass by them.  You want to minimize the fuel that it takes to “accelerate” your shuttle along a curved path passing through the 5 points.  What would be your optimum path?

You can set the speed and direction you will be traveling as you fly by each of the points.  It is obvious that the speed and directions we picked in the original “Curved Path” would not give you the optimal path, it even makes a “curlycue” loop.  So how would you figure out the optimal speeds and directions that will give you an optimal curve for the shuttle?

One way is to follow a straight line between points, stopping at each point, then turning toward the next point and speeding up again. You would have to speed up, then slow to a stop and pick up an orb at each point.  To conserve fuel, you would not be able to speed up very fast between points and would end up slowly “coasting” along the lines between each of the points and you must use your fuel to stop again.  This would work, but it might be slow and boring, especially if the points are spread far apart.

The way that would be most fun is to build up some speed and figure out the optimal way to change direction so that your net will scoop up the orbs on the fly as you pass by them.  Keep in mind that every time you change direction, or accelerate, you are using fuel.  Coasting in a straight line, with zero acceleration, does not cost you any fuel.

So, the idea is to minimize your acceleration along your whole path (x(t), y(t)).  Recall that your direction and speed (velocity “v”) is determined by the derivatives of your position along the path:

  (v_x(t), v_y(t)) = (dx/dt, dy/dt)

Your acceleration “a” is determined by how fast your velocity is changing.  And so, the derivative of your velocity is your acceleration:

(a_x(t), a_y(t)) = (dv_x/dt, dv_y/dt).

Now comes the key innovation: to save time and typing, we are going to start using the notation of “linear algebra” to solve the problem.  We will use bold-faced letters (p, v, a) (position, velocity, acceleration) to denote these “vector” entities, with the understanding that these bolded letters represent the whole “vector” consisting of all the components:

p(t) = (x(t), y(t))
v(t) = (v_x, v_y)
a(t) = (a_x, a_y)

Now we take this a step further, we will use the same “cubic” basis we used in “Building a Curved Path” to write out the curve, we will express this as the vector “b”:

b(t) = (1, t, t², t³)

and using the “dot product” notation we write the equations:

x(t) = (c_x1, c_x2, c_x3, c_x4)^.(1, t, t², t³) = c_x^.b(t)
y(t) = (c_y1, c_y2, c_y3, c_y4)^.(1, t, t², t³) = c_y^.b(t)

where the “c_x, and c_y” vectors of coefficients that determine the shape of the curve. Please refer back to the “Building a Curved Path” post for clarity.  So, the shortened notation for the equations of the curve is:

p(t) = (x(t), y(t)) = (c_x^.b(t), c_y^.b(t)) = [c_x, c_y]^.b(t)

There is conserved complexity in this simplified notation.  First of all, the [c_x, c_y] is a vector of vectors, representing both the 4 coefficients for the x(t) (c_x) and the 4 coefficients for the y(t) (c_y) functions. (8 coefficients in all).  Now you can see why we save a lot of typing with this notation.

[c_x, c_y] (a vector of vectors) is also referred to as a “matrix” the 4 c_x coefficients are sometimes written as the first row and the 4 c_y coefficients as the second row, so this “matrix” is referred to as a 2×4 matrix (C).  And again, by conservation of complexity, we can also write matrixes as bold capital letters:

C =  [c_x, c_y]

Thus, the equation for the curve can be written in matrix form simply as:

p(t) = [c_x, c_y]^.b(t) = C^.b(t)

and we just need to find the 8 coefficients in the C matrix to completely determine the curve.

C is a 2×4 matrix and b(t) is a 4×1 vector.  This notation not only makes things easier to write down, but it turns out to be very helpful (similar to the reason we used the “dot product” notation).

Another reason this is so helpful, is because if you want to take derivatives you just need to find the derivatives of b(t), which is more simple than taking derivatives of x(t) or y(t):

b(t) = (1, t, t², t³)
db/dt = (0, 1, 2*t, 3*t²)
d²b/dt² = (0, 0, 2, 6*t)

The expression d²b/dt² means the “second derivative” of b, or the derivative of the derivative.

So, since the C matrix does not change when you take derivatives only on the b(t) vector,

p(t) = (x(t), y(t)) = C^.b(t)
v(t) = (v_x, v_y) = C^.(db/dt)(t)
a(t) = (a_x, a_y) = C^.(d²b/dt²)(t)

and we have found the linear-algebra equations for three important characteristics of this curve, the position, velocity, and acceleration at every point “t” along the curve.

Note that we have not changed the equations, just how we write them down. Believe it or not, this notation makes it much easier to find the unknown coefficients in the matrix C that defines the curve.  First we must specify where we want the curve to go.

Again, we set the condition that when t=0, the curve starts at the point (x₀, y₀).  we also set the velocity at t=0 to be (v_x0, v_y0).  Now we then have these 4 known numbers, we need 8 unknown numbers (in the matrix C).  From the principles of “linear algebra” if we have 8 known numbers, we can find the 8 unknown numbers in C. We still need 4 more known numbers.

We can now set the ending point to be (x₁, y₁) and the velocity to be (v_x1, v_y1) when t =1.  These are the 4 more conditions (known numbers) that we need.  These conditions at the “endpoints” are called the boundary conditions.  If we have 8 boundary conditions, then we generally can solve for the 8 unknown coefficients.

Here is how: First, write down the equations for the boundary conditions:

p(0) = (x(0), y(0)) = (x₀, y₀) = C^.b(0)
p(1) = (x(1), y(1)) = (x₁, y₁) =  C^.b(1)
v(0) = (v_x0, v_y0) = C^.(db/dt)(0)
v(1) = (v_x1, v_y1) = C^.(db/dt)(1)

Then write these boundary points (Bp) in matrix form:

Bp = [(x₀, x₁, v_x0 v_x1), (y₀, y₁, v_y0, v_y1)]
=  C^.[b(0), b(1),  db/dt(0),  db/dt(1)]    (substitute for t)
= C^.[(1,0,0,0), (1,1,1,1), (0,1,0,0), (0,1,2,3)]
= C^.B (where B is the matrix […] above)

So these boundary conditions boil down to an equation of matrixes:

Bp =  C^.B

We know Bp, it is a 2×4 matrix of boundary points and velocities.  We know B, it is a 4×4 matrix of basis functions and their derivatives at the boundary endpoints (t=0, t=1).  We just need to find the C matrix (2×4).  It wouldn’t it be nice if we could somehow “divide” by B so we can determine what the C matrix is.

It turns out that linear algebra gives us a way to do this by finding the inverse matrix of B, called B^-1.  Inverse matrixes can be found on many calculators and computers. If you “dot” a matrix by its own inverse you get the “identity” matrix Id (we discussed earlier).  Dotting anything with the identity matrix is just like multiplying by number 1, anything dotted with the identity matrix just returns itself:

B^.B^-1 = Id = [(1,0,0,0), (0,1,0,0), (0,0,1,0), (0,0,0,1)]

In this case the calculator shows (check it out):

When
B = [(1,0,0,0), (1,1,1,1), (0,1,0,0), (0,1,2,3)]
then
B^-1 = [(1,0,-3,2), (0,0,3,-2), (0,1,-2,1), (0,0,-1,1)]

And

C^.Id = C

So dotting the above equation (Bp =  C^.B with B^-1) on both sides gives you the C matrix:

Bp^.B^-1 = C^.B^.B^-1 = C^.Id = C

We have found C, so now we can now write the equation for the curve:

(x(t), y(t)) = C^.b(t)
= (Bp^.B^-1)^.b(t)
= Bp^.(B^-1.b(t))
= Bp^.g(t)

And yet again we have defined a new vector:

g(t) = B^-1.b(t)

called a “Green’s Function”.  Once we have this Green’s Function g(t), we just “dot” it with the Boundary matrix Bp and get the curve that fits these boundary conditions.  Let that soak in for a while.  All you have to do is change the Boundary conditions matrix Bp (the beginning and ending points and velocities) dot it with g(t) and you will automatically get the equations for the curve that “magically” matches these boundary conditions (at t=0 and t=1).  All of the “hard work” needed to find the C coefficients is contained in the Green’s function.  This makes drawing and hooking together the curves easy, if we know what the points and velocities are at the endpoints of the curve.  Let’s list out the Green’s function:

g(t) = (1-3*t²+2*t³, 3*t²-2*t³, t-2*t²+t³, -t²+t³)

This same Green’s Function works for all dimensions. It is easy to see how to extend this to 3-Dimensions, just put the “z” boundary conditions into the Bp matrix and it solves for z(t).

Up to this point, in solving our problem, we have made some great progress with linear algebra notation, but we still have not found the optimal velocities (speed and direction) through the points that will minimize the acceleration along the path.  However, I think that this is enough for now.  Let’s take a breather.  It might take a while for the “linear algebra” notation we have learned to make sense.

I encourage you to write out the matrixes and see specifically how these dot products work. We will continue to solve the optimal curve in our next post.

While we are here, it might be fun and instructive to work out the math for the trajectory of a falling object using the calculus tools we have developed.

Experiment has shown that if you drop an object, it starts out at 0 velocity and starts moving downward so that after 1 second it is traveling at 9.8 meters per second (m/s) downward, after 2 seconds it is traveling at 19.6 m/s downward (exactly twice as fast).  And so, the velocity in the downward direction (negative y-direction) is

  v_y(t) = -(9.8 m/s/s) * t

where “t” is the time it has been dropping.

We often use the letter “g” to be the acceleration of gravity g=9.8m/s/s to show how fast things speed up when they are dropped. It is measured in meters per second per second.

Thus, when we drop an object, we then can calculate the speed that it will be falling at every moment “t”:

v_y(t) = -g * t

We want to find the position of how far it has dropped at every moment “t”:

v_y(t) = dy/dt = -g * t
y(t) = ?

We can use differentials and the Integral function from the previous post to figure this out, we drop the object at some height when t=0, “y(0)”, and want to know the height at any time “t”, “y(t)”:

dy = -g * t * dt

I_y(0)^y(t)(dy) = I₀^t(-g*t*dt)

y(t) – y(0) = I₀^t(-g*t*dt)

We can find this Integral if we know the Anti-Derivative of -g*t:

AD(-g*t) = -g*t²/2 + C

Which you can check by taking its derivative again.  And so, we can solve the integral I₀^t(-g*t*dt):

y(t) – y(0) = AD(-g*t)(t) – AD(-g*t)(0)

y(t) – y(0) = (-g*t²/2 + C) – (0 + C)
y(t) – y(0) = -g*t²/2     or
y(t) = y(0) – g*t²/2

which gives you the y(t) coordinate of the object that you dropped from a height y(0) at all values of “t”.  This is proven out by experiment.

Let us figure out Newton’s problem of finding the trajectory (x(t), y(t)) of a cannon ball that is shot in a direction (D_x, D_y) at a speed “S” at the time “t=0”.  We set up the coordinate so that “y” is pointing in the upward direction and “x” is pointing horizontally in the direction the cannon is fired.  We know the velocity at t=0, and the position of the cannon is (0, 0):

(v_x(0), v_y(0)) = (D_x, D_y)*S
(x(0), y(0)) = (0, 0)

So since

(v_x(t), v_y(t)) = (dx/dt, dy/dt)

And the velocity in the x-direction “v_x” does not change v_x(t) = v_x(0) and the velocity in the y-direction is always decreasing v_y(t) = v_y(0)-g*t:

(dx, dy) = (v_x(0)*dt, (v_y(0)-g*t)*dt)

We integrate both sides to get:

(x(t)-x(0), y(t)-y(0)) = (v_x(0)*t, v_y(0)*t–g*t²/2)

(x(t), y(t)) = (x(0)+v_x(0)*t,  y(0)+v_y(0)*t–g*t²/2)

Finally substituting in our starting values from above:

(x(t), y(t)) = (D_x*S*t,  D_y*S*t – g*t²/2)

We have arrived at the trajectory of the cannon ball at every time “t”.  It is a simple formula that depends only on simple math that is easy to calculate.

At this point, you might sense that we can in theory solve for the trajectory of any object if we know the acceleration that is acting on it at any point in space.  This field of study is called kinematics, and it is used to figure out the trajectory of planets in orbit, or trajectories of rocket ships floating through space, or basically the path of any other object in the universe.  This is also what must have been floating through Newton’s head when he realized that these principles were universal.

Finally, we have all the tools laid out and ready, we can focus on doing something constructive, or at least interesting, with the math we have learned.

Often in architecture, engineering, physics, math, chemistry, design, artwork, or any number of creative fields, we need to build paths (lines and curves) to describe motion or outline objects.

We already have drawn lines and hooked them together to make straight “angular paths”, like we did in orienteering, going in straight lines from point to point.

We start with a bunch of points placed in the order that we want our path to go through, P₀, P₁, P₂, P₃, P₄.  We listed 5 here, but there could be any number of points on our path.  We will also start by looking at just the “x” coordinate of the first segment between the first two points, P₀ and P₁.  As we have seen, we can expand this into more dimensions (x, y, z) later.

Let’s look at the, now familiar, equation for the x coordinate of a line between the first two points P₀:(x₀, y₀) and P₁:(x₁, y₁):

    x(S) = x₀ + D_x0*S

The independent variable “S” marks the distance traveled along this segment from P₀, and the D_x0 is the x-direction of this segment between P₀ and P₁.  “S” starts at S=0 (at P₀) and goes to S=S₁ (at P₁), where “S₁“ is the distance of the line segment between P₀ and P₁.

Since this first segment of the path stops when the line reaches the end point (x₁, y₁) after traveling the distance between them, S₁:

     x₁ = x(S₁) = x₀ + D_x0*S₁

thus

     x₁ – x₀ = D_x0*S₁        and
     D_x0 = (x₁ – x₀)/S₁

Substituting this expression for D_x0 into the equation of the line we get:

     x(S) = x₀ + (x₁ – x₀)*S/S₁

Notice that the ratio S/S₁ can be thought of as a percentage and ranges from 0 to 1 as S goes from 0 to S₁.  To avoid calculating lengths of the segments, we can use the substitution:

        t = S/S₁

into the equation for a line and we get:

       x(t) = x₀ + (x₁ – x₀)*t     where t goes from 0 to 1.

This turns out to be a very convenient way to write the equation of a line between two points.  With this equation, x(0) = x₀ and x(1) = x₁, and “t” marks the “percentage” traveled along the line segment, from 0% (t=0) to 100% (t=1).

Expanding this into two dimensions we get the equations for all 4 segments between the 5 points:

Segment 1:      (x(t), y(t)) = (x₀, y₀) + (x₁-x₀, y₁-y₀)*t
Segment 2:      (x(t), y(t)) = (x₁, y₁) + (x₂-x₁, y₂-y₁)*t
Segment 3:      (x(t), y(t)) = (x₂, y₂) + (x₃-x₂, y₃-y₂)*t
Segment 4:      (x(t), y(t)) = (x₃, y₃) + (x₄-x₃, y₄-y₃)*t

In each case, “t” goes from 0 to 1.  This is written out in full detail to make sure that you can see the pattern.  In two dimensions you can find the length of Segment 1 by S₁² = (x₁-x₀)² + (y₁-y₀)².  The others are not written out, the pattern should be obvious.

Alright, we can draw lines between the points. Now let’s look at drawing curves between the points.  This becomes a bit harder to pin down because there literally are an infinite number of curves that could be drawn between these same 5 points.  We will have to specify the kind of curves we want.

We will learn how to draw curve segments between the points and put them together smoothly to form any continuous path we desire.  We could even use this math to find an equation that describes the path of a fly walking around on the ceiling.

One common way of doing this is to use a method called “cubic splines”.  With this method the coordinates of each curve are written as a “cubic” equation in t:

    x(t) = c₀ + c₁*t + c₂*t² + c₃*t³

where we can pick any numbers we want for c₀, c₁, c₂, and c₃.  The numbers we pick determine the shape and position of the curve. We can also write this expression using the “dot” product we have discussed before:

    x(t) = (c₀, c₁, c₂, c₃)^.(1, t, t², t³)

The “c”s are called the coefficients and the powers of “t” are called the basis functions.  We can now take the derivative of the x-coordinate as we learned in the last post.

      dx/dt = (c₀, c₁, c₂, c₃)^.(0, 1, 2*t, 3*t²)

as you recall, we can use these derivatives to find the direction of the tangent line at any point “t” along the curve.  As it turns out, we now have all the tools necessary to draw a “cubic spline” curve between the points.

We can pick “t” to be a “percentage of the curve” where “t” starts at t=0, the beginning of the curve, and goes to t=1 at the end of the curve (similar to how we drew lines).  And so:

    x(0) = (c₀, c₁, c₂, c₃)^.(1, 0, 0, 0)  =  x₀ (Substitute t=0 into the equations above)
    x(1) = (c₀, c₁, c₂, c₃)^.(1, 1, 1, 1)  =  x₁ (Substitute t=1 into the equations above)

written out this means:

    c₀ + 0  + 0 +  0  = x₀           and
    c₀ + c₁ + c₂ + c₃ = x₁

and we have found that the coefficient c₀ = x₀ (the x-coordinate of the starting point) and the sum of the coefficients must add to x₁ (the x-coordinate of the ending point).  This meets the conditions that the curve will start at the beginning point and end at the ending point.  We still need some more conditions to determine what the other c’s will be.

It is natural to also place conditions on the direction that the curve leaves the starting point and the direction it arrives at the ending point.  This is determined by the tangent direction (derivatives) at these points:

dx/dt(0)   = (c₀, c₁, c₂, c₃)^.(0, 1, 0, 0) = S₀*D_x0
dx/dt(1)   = (c₀, c₁, c₂, c₃)^.(0, 1, 2, 3) = S₁*D_x1

We can set S₀, S₁ and the directions D_x0 and D_x1 to be whatever we want and we can immediately see:

                c₁                = S₀*D_x0
    c₁ + 2*c₂ + 3*c₃ = S₁*D_x1

We have determined c₀ and c₁ from the starting point and the direction of the tangent at that point.  We can use substitutions to find the rest.  We have 4 equations and 4 unknowns and so we do this:

      3*c₀ + 3*c₁ + 3*c₂ + 3*c₃ = 3*x₁   (multiply the sum by 3)
                       c₁ + 2*c₂ + 3*c₃ = S₁*D_x1  (from above)

Subtracting the second equation from the first and making substitutions for c₀ and c₁:

      3*c₀ +   2*c₁ +   1*c₂ + 0*c₃ = 3*x₁ – S₁*D_x1  (making subs)
      3*x₀ + 2*S₁*D_x0 + c₂              = 3*x₁ – S₁*D_x1   (solve for c₂)

          c₂ = 3*x₁ – 3*x₀ – S₁*D_x1 – 2*S₀*D_x0      or
         c₂ = 3*(x₁ – x₀) – (S₁*D_x1+2* S₀*D_x0)

          c₃ = x₁ – (c₀ + c₁ + c₂)     (rearranging the sum)
          c₃ = x₁ – [x₀ + S₀*D_x0 + 3*(x₁ – x₀) – (S₁*D_x1 + 2*S₀*D_x0)]
          c₃ = -2*(x₁ – x₀) + S₁*D_x1 + S₀*D_x0

and now we have found all the “c” coefficients.

We can now draw the curves between the points if we know the starting point (x₀, y₀), the ending point (x₁, y₁), the starting direction (D_x0,D_y0) and the ending direction (D_x1, D_y1). The S₀ and S₁ variables are arbitrary and can be varied to taste.  We now plug all of these coefficients into our original curve:

      x(t) = (c₀, c₁, c₂, c₃)^.(1, t, t², t³)
      dx/dt = (c₀, c₁, c₂, c₃)^.(0, 1, 2*t, 3*t²)   = S₀*D_x0 (tangent direction)

the y(t) equation follows exactly the same pattern (just substitute “y” for “x” in all the equations).  Voila, we have found a “cubic” curve that passes through these points and has these tangent directions.

We can now do the same for all of the curves between each pair of points, drawing curves through all of the points.  There is a little bit of wiggle room in these curve equations.  The S₀, S₁, … and direction parameters for the tangents at the points can be set at will.  Since the direction vectors are the same at the points, the curve will smoothly pass through the points in the same direction.  These direction vectors show the “nose” direction (D_x,D_y) and “speed” (S) that a fly would be traveling if it were walking along the curve. It is fun to program these equations into a computer that draws them and watch the curves change as you change the location of the points and the directions and magnitudes of the tangents at the points.  This is how most modern, curve-fitting software works.

Of course, you can use another set of “basis functions” rather than just the powers of “t”, if you wish, to possibly get a better fit on your curves.  You might also want to impose other conditions, such as minimum curvature.  Just remember, if you use more basis functions, you will have more equations to solve for, you will need more conditions, and you might need some experience in “linear algebra” to solve for the coefficients.  Happy curving!

We covered the concept of differential calculus in just the last post.  It is time to slow down just a bit, catch our breath, and explore more in depth what this all means.

First, we started out with the concept of a function.  We used the example of a curve in space where the x-coordinate (and all of the others) depend on an index “t” that indicates where each point is along this curve in space.

     x(t) = x-coordinate of a point at the index t

If you specify a “t” value, the function x(t) will give the x-coordinate of a point along the curve that corresponds to “t”.

It might not have been perfectly clear, but we assumed that the curve is made out of points that are infinitely close to each other (same as a line).  There are no gaps or kinks in the curve, such that when you increase “t” by an infinitesimally small amount “dt” it will return the coordinates of a point that is infinitesimally close along the curve.  This is the concept of a “smooth” or “continuous” function.

An example of this is a car traveling down a straight road.  Suppose the function x(t) shows how far the car has gone down the road at a time “t”.  At each time, “t”, the function x(t) will tell you the exact position of the car. Now since a car can’t instantaneously disappear from one point and appear in a new point, it always takes time to travel continuously between any two points, which means that if you look at the position of the car a billionth of a second later, the position will be very close to where it was before.

    As we discussed, “dt” is much smaller than a billionth of a second, it is the smallest amount possible without being zero.  If “t” has increased by a small amount, “t+dt”, then the x-coordinate also will have changed by a very small amount “dx”. This is what we mean by the expression:

           dx = x(t+dt) – x(t)

The derivative dx/dt is defined to be:

     dx/dt  = [x(t+dt) – x(t)]/dt

which indicates how fast “x” is changing in relation to a change in “t” at the point “t”.  We can now hopefully see why differential calculus is the mathematics of motion.  In the example of the car above, dx/dt is velocity of the car at any instant “t” along its path – it moves a distance dx in the time dt.

    In the example where:

    x(t)       =      t²
 x(t+dt)   =  (t+dt)²

   dx = x(t+dt) – x(t)
         =  (t+dt)² –  t²
         = t²+2*t*dt + dt² – t²   (the t² cancels out)
         = 2*t*dt + dt²

and

    dx/dt = [x(t+dt) – x(t)]/dt
                 = 2*t       (when dt –> zero)

We say that the derivative of t² is 2*t, which also indicates how fast x is changing in ratio to the change in t at every point along the curve.

This function and differential notation is very powerful, we can get the answers to some questions without even knowing the exact function.

Suppose that x(t) is the sum of two functions of t, f(t), g(t):

    x(t) = f(t) + g(t)    where f(t) and g(t) are smooth

     df = f(t+dt) – f(t)
     dg = g(t+dt) – g(t)

dx =         x(t+dt)                –       x(t)
         = f(t+dt) + g(t+dt)     –    (f(t) + g(t))
         = [f(t+dt) – f(t)] + [g(t+dt) – g(t)]
         =             df           +          dg

Thus the derivative is also the sum of the derivatives:

    dx/dt =  df/dt  + dg/dt.   

Suppose that x(t) is the product of two functions of t:

    x(t) = f(t)*g(t)    where f(t) and g(t) are smooth

   dx =           x(t+dt)          –       x(t)
         = f(t+dt)*g(t+dt)     –    f(t)*g(t)
         =[df + f(t)]*[dg + g(t)] – f(t)*g(t)
         = df*dg + df*g(t) + f(t)*dg + f(t)*g(t) – f(t)*g(t)
         = df*dg + df*g(t) + f(t)*dt

Thus the derivative is:

dx/dt = df*(dg/dt) + (df/dt)*g(t) + f(t)*(dg/dt)
            = (df/dt)*g(t) + f(t)*(dg/dt)     because df –> 0

dx/dt = (df/dt)*g(t) + f(t)*(dg/dt)

Notice that we must always divide differentials by differentials to get derivatives, and all differentials standing alone (dx, df, dg, dt) go to zero.

We can now use the product rule to get the derivative of:

    x(t) = t³ = t² * t   {f(t) = t²,  g(t) = t}

we already know that df/dt = 2*t from above and

    dg/dt = [t+dt – t]/dt = 1

therefore since

dx/dt = (df/dt)*g(t) + f(t)*(dg/dt)

dx/dt =    (2*t)*t      +    t²*1
                  = 3*t²

By extending this we can show that for any power of t:

If x(t) = tⁿ  =  t^(n-1) * t

then

  dx/dt = n * t^(n-1)

We can see where some of the derivative tables in your calculus book come from.

We will end off by finding the derivative for a ratio of functions:

    x(t) = f(t)/g(t)

   dx/dt = [f(t+dt)/g(t+dt)        –    f(t)/g(t)]/dt
               = [(df+f(t))/(dg+g(t)) –    f(t)/g(t)]/dt
               = [[g(t)*(df+f(t))-f(t)*(dg+g(t))]/(g(t)*(dg+g(t))]/dt
  = [g(t)*(df/dt) + f(t)*g(t)/dt – f(t)*(dg/dt) – f(t)*g(t)/dt]/ (g(t)*dg + g(t)*g(t))

and when df and dg –> 0,

    dx/dt  = [g(t)*(df/dt) – f(t)*(dg/dt)]/(g(t)*g(t))

When we learn the “algebra” of differentials, we can find derivatives for a vast array of smooth functions.

As we have seen, when we know the derivatives of a curve, we can find the direction of the tangent line at any point along the curve. Recall from the last post in 2-D:

    (D_x, D_y) = (dx/dt, dy/dt)/((dx/dt)²+(dy/dt)²)^1/2

The nice thing about this notation is that it is easy to go to 3-D, 4-D, and as many -D’s that you want just by continuing the pattern.

Calculus deals with the mathematics of movement. In order to move from a starting point (x(S), y(S)), in 2-D, to the point next to it, you need to know how to move a tiny distance “dS” in some direction (D_x, D_y).  If we are moving along a line, then we already have a way to express this.  If (x(S+dS), y(S+dS)) is the function notation for the adjacent point along the line

(x(S), y(S)) = (x₀, y₀) + (D_x, D_y)*S

(we are writing this in function notation) then the line segment that connects them can be written (it works out):

       (x(S+dS), y(S+dS)) = (x(S), y(S)) + (D_x, D_y)*dS

Looking at just the x-coordinate this means:

     x(S+dS) = x(S) + D_x*dS   or

     x(S+dS) – x(S) = D_x*dS

Now it is pretty easy to see that when dS goes to zero, the distance between the x-coordinate of the ending point and the starting point  “x(S+dS) – x(S)” also goes to zero.  We can use our differential notation to express this:

       dx = x(S+dS) – x(S)

where “dx” is defined as the difference between the x-coordinates.  We now can write the differential expression:

       dx = D_x*dS

Recall that the small “dx” means differential (the infinitesimal distance moved, and the large “D_x” means the direction it moves.  See the diagram above.

Now we are ready to make the leap.  Even though dx and dS get as close as possible to zero, their ratio in this example is fixed:

                    dx/dS = D_x

This is because the Direction of the line segment is fixed and does not depend on the distance between points, no matter how small.

The concept of calculus is used to travel a tiny distance “dS” to arrive at an adjoining point.  No matter how small dS is, you can always cut it in half (dS/2) to find a point that is closer.  Recall that this concept allows us to “zoom in” to a single point throughout all eternity and never get to the point.  In calculus “dS” is called a “differential” and it is “infinitesimally” small (as close as possible to zero, without being zero).  This is the concept that makes Calculus a bit strange but also makes it interesting.

We can use the same concepts to describe what happens along a smooth non-linear curve (x(t), y(t)).  If we then move along the curve by increasing “t” by “dt”, an infinitesimal amount to the point (x(t+dt), y(t+dt)) then these two points on the curve are infinitesimally close to one another.  We define the differentials:

     (dx, dy) = (x(t+dt)-x(t), y(t+dt)-y(t))  or

        dx = x(t+dt) – x(t)

        dy = y(t+dt) – y(t)

Note that the variable “t” no longer denotes a distance, it is just an “index” that determines the position of the point on the curve.

Let’s consider an example, with just the x-coordinate.  Suppose

     x(t) = 1+ t²   then

     x(t+dt) = 1+ (t+dt)² = 1+ t² +2*t*dt + dt²

      dx = x(t+dt) – x(t) = 2*t*dt + dt²

      dx/dt = 2*t + dt

as dt goes to zero, we can ignore the “dt” term.

        dx/dt = 2*t

The line between these two points can be written as:

       (x(t+dt), y(t+dt)) = (x(t), y(t)) + (D_x, D_y)*dS

      (x(t+dt)-x(t), y(t+dt)-y(t)) = (D_x, D_y)*dS

Or   (dx(t), dy(t)) = (D_x, D_y)*dS

now dS² = dx(t)² + dy(t)² or

          dS² = [(dx(t)/dt)² + (dy(t)/dt)²]*dt²

which shows that “dS” also scales with “dt”.

Once we know these relationships then we can solve for the direction (D_x(t), D_y(t)) of the line that connects two infinitesimally close points on the curve (x(t), y(t)) at the index “t”.  We use the function notation for the direction parameters to indicate that the direction of this “tangent” line depends on where it is along the curve.

Just as in the case of a line, even if dS, dt, dx, and dy go to zero, as the points get closer together, they could have ratios between them that do not go to zero.  Everything depends on the shape of the curve.  Certainly, we expect that the direction of the line will not necessarily change as the points converge.  Such a curve is called “differentiable”.

Let’s look at an example.  Suppose:

   (x(t), y(t)) = (t, t²)              then

   (x(t+dt), y(t+dt)) = (t+dt, (t+dt)²) = (t+dt, t²+2*t*dt + dt²)

   (dx(t), dy(t)) = (x(t+dt)-x(t), y(t+dt)-y(t)) = (dt, 2*t*dt+dt²)

    dS² = dx² + dy² = dt²+(2*t*dt)² = [1+(2*t)²]*dt²

since    (dx(t), dy(t)) = (D_x, D_y)*dS

              (dt, 2*t*dt+dt²) = (D_x, D_y)* [1+(2*t)²]^1/2*dt

So    (D_x, D_y) = (dt, 2*t*dt+ dt²)/ [1+(2*t)²]^1/2*dt

and  (D_x, D_y) = (1, 2*t + dt)/ [1+(2*t)²]^1/2

And so, the direction of the line is well defined when “dt” goes to zero, and we add function notation:

     (D_x(t), D_y(t)) = (1, 2*t)/ [1+(2*t)²]^1/2

This line direction is called the tangent to the curve at the point (x(t), y(t)).  At this point, this line will only touch the curve at this one singular point.

Also

   dS/dt = [1+(2*t)²]^1/2

and all is solved for every location “t” on the curve.  Perhaps there are many questions remaining.  One of them being, for example, how to calculate the distance along the curve “S” from one point to another.  These questions will be left for other posts.

So far, we have only described lines where the distance between any two points on the line is well defined by a “metric”.  Before we look at how to draw curves, let’s remind ourselves what a line looks like in math language:

     (x, y) = (x₀, y₀) + (D_x, D_y)*S

The starting point on the line is (x₀, y₀) and the direction the line takes (D_x, D_y) in a cartesian coordinate system, and “S” is the distance from the starting point. Note that “S” serves as an index or indicator for every point on the line. In math language, “S” is known as the “independent variable”. It determines the exact point on a line a distance “S” from the starting point. Of course, we assume we already know the starting point (x₀, y₀) and direction (D_x, D_y) of the line.

In “function notation” we put the independent variable (S) next to the “dependent variables” (x and y) in parenthesis to remind ourselves that if we pick a number for “S”, it will give us an exact point (x(S), y(S)) on the line:

     (x(S), y(S)) = (x₀, y₀) + (D_x, D_y)*S

This “function” notation is extremely helpful. We can use this notation to replace the variable “S” with any number or symbol we wish. For example, let us replace the variable “S” with “t” (like a “cut-paste” of “S” with “t”):

     (x(t), y(t)) = (x₀, y₀) + (D_x, D_y)*t

We can even replace “S” with a mathematical statement:

     (x(S+3), y(S+3)) = (x₀, y₀) + (D_x, D_y)*(S+3)

The rule is that whatever goes in the parenthesis in function notation it is substituted into the “function” at the appropriate place.

Now we can start drawing curves by considering functions that are “non-linear” (they don’t draw a line).  We have already looked at how to draw a circle.  You may recall that we use the same equation for a line, but instead of varying the distance “S” we varied the direction (D_x, D_y) = (Cos(t), Sin(t)), where “t” is the angle off the x-axis.  So, the equation for a circle of radius S=1, centered at (x₀, y₀) = (0, 0) would be written:

        (x(t), y(t)) = (Cos(t), Sin(t))

where “t” goes from 0 to 360 degrees.  We could also draw just any arc of the circle by determining the range that the angle “t” spans, for example, a quarter circle is drawn when t goes from 0 to 90 degrees.  You get the idea.

We should mention at this point that we chose “t” as the “index” that indicates where any point is on the curve.  It doesn’t tell us how much distance traveled along the curve, or even the time that it takes for a point to travel along the curve.  It is just taken as a general parameter.  In the case of the circle of radius 1, we know that the circumference is 2*Pi over 360 degrees and so the distance traveled along the circle from 0 to the angle “t” is S = 2*Pi * (t/360).

Now we can have fun drawing a bunch of curves, using this function notation.

An oval that has major axis 3 and minor axis 2 has this equation:

        (x(t), y(t)) = (3*Cos(t), 2*Sin(t))

A spiral that starts at (0, 0) and goes to (1, 0) is drawn by this equation:

        (x(t), y(t)) = ((t/360)*Cos(t), (t/360)*Sin(t))

We can also draw curves in 3-dimensions just as easily; here is a spiral around a cone:

      (x(t), y(t), z(t)) = ((t/360)*Cos(t), (t/360)*Sin(t), t/360)

We can use any non-linear equation to draw a curve, like:

(x(t), y(t)) = (-2*t+3*t², 2*t²-5*t³+4*t⁴)

We could keep going all day, curves are always visually interesting.

Later we will see how we can hook curves together to form any shape we want, but first we need another tool to analyze curves, differential calculus.  It really is not as difficult as you might think, so keep tuned.

As we enter dimensions beyond our physical experience, it might help our understanding to look back and see patterns in smaller dimensions that still apply in larger dimensions.  Is there such a pattern when finding directions (D_x, D_y, D_z, D_t) in 4 dimensions or larger.  One obvious pattern, as dimensions increase, is the way we measure distance:

   1-D:         D_x² = 1
   2-D:         D_x² + D_y² = 1
   3-D:          D_x² + D_y² + D_z² = 1
   4-D:          D_x² + D_y² + D_z² + D_t² = 1

The pattern for higher dimensions is obvious. We expect this pattern to continue to any number of dimensions; we just keep adding on perpendicular triangles.  See the diagram above.

There is one problem with setting directions in higher dimensions that is not so obvious. For example, in 1-D, D_x² = 1 means that D_x = 1 or D_x = -1, this defines the directions to be along the positive or negative x-axis.  So far so good. In 2-D, however, when D_y² = 1, it forces D_x² = 0, which does not indicate either a positive or negative direction along the x-axis.  This ambiguity is due to the rotational symmetry of perpendicular lines, as we have mentioned before, you can “flip” negative and positive directions around a perpendicular direction, and nothing changes.

This ambiguity of direction exists whenever we add a new dimension.  In 3-D, for example, when D_z² = 1, it forces D_x² + D_y² = 0 and there are ambiguities as to where to place the direction of both the x-axis and y-axis.  To a man standing exactly on the north pole, for example, every direction is south, there is no east and west.  I have also heard it said that no matter how you comb the fuzz flat on a tennis ball, there will always be a part (singularity).

To deal with these ambiguities when any of our direction parameters is zero, it is helpful to fix the angles of the x-axis, y-axis, z-axis, etc. so that directions will not be ambiguous when you pass through the “singularities” where one or more of the direction parameters are forced to be zero.  So a man standing on the north pole, where every direction is south, still knows which angle to face if he wants to point to Paris, France.  This makes defining directions consistent.

Let us look at a way to define directions as we are increasing dimensions. We defined the angle A_xy to be the angle from the x-axis in the x-y plane.  Then we defined the angle A_z to be the angle off of the x-y plane in the z direction.  Listing the direction parameters for the dimensions we know:

   2D: D₂ = (D_2x, D_2y) = (Cos(A_xy), Sin(A_xy))
   3D: D₃ = (D_3x, D_3y, D_3z) = (D_2x*Cos(A_z), D_2y*Cos(A_z), Sin(A_z))
                 = (D_2x, D_2y, 0)*Cos(A_z) + (0, 0, 1)*Sin(A_z)

We added the “2” and “3” subscripts to indicate the dimension.

It might take you a few minutes to see the pattern.  Expanding into 4-D, we define the angle A_t to be the angle off the x-y-z space toward the t-axis, then continuing the pattern, the direction in 4-D is:

    D₄ = (D_4x, D_4y, D_4z, D_4t) = (D_3x, D_3y, D_3z, 0)*Cos(A_t) + (0, 0, 0, 1)*Sin(A_t)

    So putting them together and stacking them we get:

         D_4x = Cos(A_xy)*Cos(A_z)*Cos(A_t)
         D_4y = Sin(A_xy)*Cos(A_z)*Cos(A_t)
         D_4z =                   Sin(A_z)*Cos(A_t)
         D_4t =                                   Sin(A_t)

And we have found the Direction parameters in 4-D, and hopefully we can see the pattern that will allow us to use the same scheme to find directions in any dimension. An interesting side note: The angle in the t-direction, A_t, determines how close the 4-D direction is to the t-axis. When it gets close to 90 degrees, the direction is close to the t-axis and Cos(A_t) is close to zero, and D_4x, D_4y and D_4z are close to zero, this restricts motion in the x-y-z space. The faster time is moving, the more restricted the motion is in 3-D space. How strange. We will see just how strange in the following post.

We can navigate in 3-D to any point (x_R, y_R, z_R), in Cartesian Coordinates, by starting at the origin (0, 0, 0) and in the 2-D, x-y plane, walking in the x-direction (1, 0, 0) a distance x_R, then turning left 90 degrees and walking a distance y_R in the y-direction (0, 1, 0), still on the x-y plane.  This is the same as we did in 2-D except we put an extra ‘0’ in the third ‘z’ direction.  At this point we are still on the x-y plane at the point (x_R, y_R, 0).  The z-direction is straight up, so we would now need a ladder to climb up or a shovel to dig down, it is helpful to pretend we can fly (or dig).  From this point (x_R, y_R, 0) on the x-y “ground” plane, we fly straight up in the z-direction (0, 0, 1) a distance z_R, and we successfully arrive at the point (x_R, y_R, z_R).

We can express this journey with vectors, as we did in 2-D, by connecting (adding) lines together to form the path:

   (x_R, y_R, z_R) = (1, 0, 0)*x_R + (0, 1, 0)*y_R + (0, 0, 1)*z_R
                        = (x_R, 0, 0) + (0, y_R, 0) + (0, 0, z_R)
                        = (x_R, y_R, z_R)

        And using dot product notation, this can be written:

   (x_R, y_R, z_R) = (1, 0, 0)*x_R + (0, 1, 0)*y_R + (0, 0, 1)*z_R
                        = [(1, 0, 0), (0, 1, 0), (0, 0, 1)]^.(x_R, y_R, z_R)

The [(1, 0, 0), (0, 1, 0), (0, 0, 1)] matrix is called the identity matrix.

We could also arrive faster by flying along a straight line connecting the two. Fly along a direction (D_x, D_y, D_z), from the origin directly to this point, (x_R, y_R, z_R) a distance “R” away:

   (x_R, y_R, z_R) = (D_x, D_y, D_z)*R

        Where the distance ‘R’ is found by:

    R² = x_R² + y_R² + z_R²

From the equation above for this line, we can easily find the direction vector from the origin to any point (x_R, y_R, z_R), similar to what we did in 2-D.

      (D_x, D_y, D_z) = (x_R/R, y_R/R, z_R/R)

Recall that the direction vector always has a distance of 1 unit, as seen in the diagram above.  In the diagram, we have attempted to show a 3-D path on a 2-D piece of paper.  Note that in the diagram, D_r is the distance from the origin to the point (D_x, D_y, 0) in the x-y plane. From the Pythagorean theorem, we find that the length:

      D_r² = D_x² + D_y²    and
      D_r² + D_z² = 1        thus combining these
      D_x² + D_y² + D_z² = 1.

The total length of the distance vector is 1 unit, as required.  Everything is consistent.  Now we use the definition of the ‘Sin’ and ‘Cos’ and can see, from the diagram above, that:

     Cos(A_xy) = D_x/D_r
     Sin(A_xy) = D_y/D_r

      Cos(A_z) = D_r/1
      Sin(A_z) = D_z/1

    Thus

       D_x = Cos(A_xy)*D_r = Cos(A_xy)*Cos(A_z)
       D_y = Sin(A_xy)*D_r = Sin(A_xy)*Cos(A_z)
       D_z = Sin(A_z)

And so, if we stand at the origin facing in the x-direction, and rotate our “head” an angle A_xy to the left, and then tilt our head an angle A_z off the x-y plane to look directly at the point (x_R, y_R, z_R), we can find the direction vector pointing to this point using these angles:

   (D_x, D_y, D_z) = (Cos(A_xy)*Cos(A_z), Sin(A_xy)*Cos(A_z), Sin(A_z))

       And then draw the line to this point:

    (x_R, y_R, z_R) = (D_x, D_y, D_z)*R

Now that we are staring at a point in the sky, with our nose pointed directly at the point, we can ask about “reference frame” of our head.  What direction is our left ear pointing, for example.  Our left ear is pointing in the direction, still in the x-y plane, that is 90 to the left of the A_xy angle we rotated our head (-D_y/D_r, D_x/D_r, 0). We divide by D_r because in 3-D we must make the length of the direction vector equal to 1.  If this seems vague, look back and review how to find 90 degree directions in the 2-D plane in the top diagram taken from previous posts.

When we tilt our head, we can ask what is the direction that the top of our head points, 90 degrees up from the angle A_z. It perhaps takes a bit more imagination to see this, since we are facing in the x-y plane (D_x/D_r, D_y/D_r, 0) and have tilted our head up to look at D_z, we find that the “shadow” of our head is pointed behind us, in the x-y plane (-D_x/D_r, -D_y/D_r, 0), then looking at the elevation D_r (when rotated 90 degrees) we find that (-D_x*D_z/D_r, -D_y*D_z/D_r, D_r) is the direction of the top of our head.  So, in the reference frame of our head:

     Direction of our Nose:     (D_x, D_y, D_z)
     Direction of Left ear:         (-D_y/D_r, D_x/D_r, 0)
     Direction of Top of head:  (-D_x*D_z/D_r, -D_y*D_z/D_r, D_r)

Let us verify that these direction vectors all have a length of 1.  The Nose direction we have verified.  For the Left ear direction:

    L^.L = (-D_y/D_r)² + (D_x/D_r)² + 0² = (D_y² + D_x²)/D_r² = 1

For the Top of head direction:

    T^.T = (-D_x*D_z/D_r)² + (-D_y*D_z/D_r)² + D_r² = (D_y² + D_x²)*D_z²/D_r² + D_r²
                                                                    = D_z² + D_r² = 1

Now let’s find what happens when we find the dot product of these direction vectors, we will call them the N, L, and T directions respectively.

N^.N = (D_x, D_y, D_z)^.(D_x, D_y, D_z) = D_x² + D_y² + D_z² = 1

N^.L = (D_x, D_y, D_z)^.(-D_y/D_r, D_x/D_r, 0) = -D_x*D_y/D_r + D_y*D_x/D_r + 0
                                                                  = 0

N^.T = (D_x, D_y, D_z)^.(-D_x*D_z/D_r, -D_y*D_z/D_r, D_r)
        = -D_x²*D_z/D_r + -D_y²*D_z/D_r + D_z*D_r
        = -(D_x² + D_y²)*D_z/D_r + D_z*D_r
        = -D_r²*D_z/D_r + D_z*D_r
        = 0

L^.T = (-D_y/D_r, D_x/D_r, 0)^.( -D_x*D_z/D_r, -D_y*D_z/D_r, D_r)
       = D_y*D_x*D_z/D_r² + -D_x*D_y*D_z/D_r² + 0
       = 0

We also verified that L^.L = 1 and T^.T = 1 above.  So, in general, if we dot any of these N, L, T direction vectors with themselves, we get ‘1’, but if we dot any of them with each other, we get ‘0’.  It turns out that this is a requirement for any three perpendicular direction vectors in a 3-D Cartesian reference frame.  I find the “head” reference frame to be more natural because we can relate to it better, so I use this reference frame for whatever direction I am looking.  It is also a great reference frame for video games.

We have already seen from perpendicular 2-D paths how we can convert between a reference frame to our “ground” x-y-z reference frame.  To understand this, let’s take a journey: Fly in the Nose direction a distance S_N, then fly in the Left ear direction a distance S_L, and then in the Top of head direction a distance S_T.  Then we will arrive at the point:

    (x, y, z) = N*S_N + L*S_L + T*S_T

       or in dot product notation:

     (x, y, z) = [N, L, T]^.(S_N, S_L, S_T)

This amazing equation lets you convert any point in your “head frame” of reference (S_N, S_L, S_T), when your Nose is pointing in a direction N:(D_x, D_y, D_z), to a point in the “ground” frame (x, y, z).  You might also want to do this the other way around.  Review how we did this in 2-D, by “dotting” each side by each of the direction vectors, we get:

    (S_N, S_L, S_T) = [N, L, T]^.(x, y, z)

This is how video games convert a 3-D world, stored in “ground” (x, y, z) coordinates, into the perspective of the viewer’s head (S_N, S_L, S_T) coordinates.  The details need to be worked out but using this equation we can map a 3-D world onto a 2-D screen in the viewer’s perspective, no matter the relative direction of the viewer’s head.  We will stop here, there is a lot to take in.