2018年5月15日 星期二

Determinants


Determinants

Ching-Tang Tseng
Hamilton, New Zealand
16 May 2018

In matrix mathematical operation, many calculations are with respected to evaluate determinant. Determinant of a matrix A can be represented by det A or det |A|.

Evaluate determinant by computer program is not a simple procedure, especially for a high rank matrix. There is a Sarrus formula could be used in evaluating the determinant of a 3x3 matrix. There are no intuitive formula could be used for a matrix if its rank is higher than 3. No intuitive formula factor makes this kind of program difficult to design.

According to a mathematical expression, the determinant of matrix A is expressed as following:

det |A| = A(1,k) * SIGN * det (cofactor(k)) = A(1,k) * (-1)**(k+1) * det (cofactor(k))    -------- (1)

Range of k is from 1 to rank n.

To design a program based on this equality has an advantage, determinant evaluating without division operation. But also has disadvantage, the programming language must owned recursive feature, owing to both sides of equality listed above has det (...) operation.

FORTH language has recursive feature but inconvenient to pass too complicated parameters, for example, something like to pass a sub-matrix det (cofactor(k)).

Fortunately, ancient able man had ever contributed other evaluating algorithm, for example, a procedure to evaluate a determinant by the leading diagonal method, using largest pivots. In this method a reduction is made matrix to upper triangular form by successive row operations. Determinant then is the product of all elements on the diagonal line. This non-recursive algorithm must using division operation, results in inevitable inaccuracy. Programming code must to do some improvements.

On many websites, in many books, we are able to find many real codes to evaluate determinants. Even though the designer of those codes did not attach their resource reference, I am sure they are absolutely not the algorithm inventor. Here, I would like to introduce an oldest resource from my personal collected books.

P. N. Corlett and J. D. Tinsley, Practical Programming Cambridge University Press, Alden & Mowbray Ltd. Oxford. Great Britain, 1968, P.185

According to this resource, I rewrote the original ALGOL code into my ABC FORTH system, all side comments included as well. This code was a resource 50 years ago. I am citing it now. It can be applied to many places again via ABC FORTH system even after 50 years.

\ A procedure to evaluate a determinant by the leading diagonal method, using largest pivots. 2008-08-23
\ Lina64 version 20180508

\ ******************************
\ (1).Data structure declared heap

6 integers N I J K P Q
2 reals a d
20 20 (MATRIX) IA
20 20  MATRIX  AA

\ ******************************
\ (2).Data input heap

\ det = 160
: INPUT-IDATA
  [[ N = 4 ]]
  [[ IA ( 1 1 ) = 1 ]] [[ IA ( 1 2 ) = 2 ]] [[ IA ( 1 3 ) = 3 ]] [[ IA ( 1 4 ) = 4 ]]
  [[ IA ( 2 1 ) = 2 ]] [[ IA ( 2 2 ) = 3 ]] [[ IA ( 2 3 ) = 4 ]] [[ IA ( 2 4 ) = 1 ]]
  [[ IA ( 3 1 ) = 3 ]] [[ IA ( 3 2 ) = 4 ]] [[ IA ( 3 3 ) = 1 ]] [[ IA ( 3 4 ) = 2 ]]
  [[ IA ( 4 1 ) = 4 ]] [[ IA ( 4 2 ) = 1 ]] [[ IA ( 4 3 ) = 2 ]] [[ IA ( 4 4 ) = 3 ]]   
;

: INPUT-DATA
  INPUT-IDATA
  basic
10 for i = 1 to N
20 for j = 1 to N
30 let { AA ( i j ) = i>r ( IA ( i j ) ) }
40 next j
50 next i
60 end ;

\ *******************************
\ (3).Auxiliary instructions heap

: SHOW-AA   
  BASIC
10 run cr
20 FOR I = 1 TO N
30 FOR J = 1 TO N
40 let { a = AA ( I  J ) }
50 run a fs. 3 spaces
60 NEXT J
70 run cr
80 NEXT I
90 END ;

\ ********************************
\ (4).Main code heap

1.0 e -12 fconstant feps

: (DET)
  BASIC
\ 10 RUN INPUT-DATA
20 LET { D = f1.0e0 }

30 FOR K = 1 TO ( N - 1 )

40 LET P = K
50 LET Q = K
60 LET { A = ABS ( AA ( K K ) ) }
70 FOR I = K TO N
80 FOR J = K TO N
90 IF {  ABS ( AA ( I J ) ) > A } THEN 110
100 GOTO 140
110 LET P = I
120 LET Q = J
130 LET { A = ABS ( AA ( I J ) ) }
140 NEXT J
150 NEXT I
\ End of search for largest element.

160 IF P <> K THEN 180
170 GOTO 230
180 FOR J = K TO N
190 LET { A = AA ( P J ) }
200 LET { AA ( P J ) = AA ( K J ) }
210 LET { AA ( K J ) = NEGATE ( A ) }
220 NEXT J
\ End of interchange of rows P and K.

230 IF Q <> K THEN 250
240 GOTO 300
250 FOR I = K TO N
260 LET { A = AA ( I Q ) }
270 LET { AA ( I Q ) = AA ( I K ) }
280 LET { AA ( I K ) = NEGATE ( A ) }
290 NEXT I
\ End of interchange of columns Q and K.
\ Largest element is now the pivot.

300 LET { D = D * AA ( K K ) }
310 IF { ABS ( D ) < feps } THEN  430
320 FOR I = K + 1 TO N
330 FOR J = K + 1 TO N
340 LET { A = AA ( K J ) * AA ( I K ) / AA ( K K ) }
350 LET { AA ( I J ) = AA ( I J ) - A }
360 IF { ABS ( AA ( I J ) ) < ABS ( A ) * feps } THEN 380
370 GOTO 390
380 LET { AA ( I J ) = f0.0e0 }
390 NEXT J
400 NEXT I
\ End of reduction to upper triangular form.

410 NEXT K

420 LET { D = D * AA ( N N ) }
430 IF {  ABS ( D ) > feps } THEN 450
440 LET { D = f0.0e0  }
450 LET { D = D }
\ Determinant is now in D
\ 460 PRINT { D }
470 END ;

\ *********************************
\ (5).Manipulation heap

: round ( f1 -- f2 )
  over abs nlog 12 > if fround then ;

: DET
  BASIC
10 RUN INPUT-DATA
20 run cr ." matrix " n . ." X " n . ." : " cr show-aa cr
30 IF N = 1 THEN 50
40 GOTO 70
50 LET { D = AA ( 1 1 ) }
60 GOTO 120
70 IF N = 2 THEN 90
80 GOTO 110
90 LET { D = AA ( 1 1 ) * AA ( 2 2 ) - AA ( 1 2 ) * AA ( 2 1 ) }
100 GOTO 120
110 RUN (DET)
120 PRINT { " determinant = " ; D }
130 let { d = d round }
140 run cr ." determinant = " d f. cr
150 END ;

det


\ ********** the end ***************

Code executed outcomes is showing as following:

matrix 4 X 4 :

1.0 E 0     2.0 E 0     3.0 E 0     4.0 E 0    
2.0 E 0     3.0 E 0     4.0 E 0     1.0 E 0    
3.0 E 0     4.0 E 0     1.0 E 0     2.0 E 0    
4.0 E 0     1.0 E 0     2.0 E 0     3.0 E 0    


determinant = 1.5999999999 E 2 
determinant = 160.0
 OK

(4).Main code heap was a typical model program. This code was my personal collection 10 years ago(2008). Code in each line had been treated as close to the original ALGOL code as possible. Other heaps were prepared in recent days to fit to my new ABC FORTH, which was added on Lina64. Obviously, word (det) had been consolidated to able to exist forever. It is easy to read and to understand. You are welcome to share this code or to rewrite it into your program language.

I posted many achievements on this website from my personal record. I am not pursued to promote my ABC FORTH now, and my floating point number represented format is thoroughly violated the traditional habits. Those points are not worth to concern, make sure useful of all my design should be concerned. The code in this article proved this insisted intention.

Integrate code with the answer was listed above for reference. The method of data input could be redesigned separately. Data input could be designed into an interactive input style or created by a separated functional data generator, or input from a separated pure text data file. The simplest demonstration is as simple as in (2).Data input heap.

At the beginning in (5).Manipulation heap, there was a round instruction designed by fround primitive instruction. fround is rounding off  last one digit of a floating point number, no matter how many digits of mantissa has. round is rounding off last one digit so long as the digits of mantissa has bigger than 12. this treatment was arranged to solve accumulated inaccuracy owing to continuous division operation. In my floating point system, exponent was always to be put on the top of data stack, mantissa was always to be put on the next secondary cell.

The real functional meaning of FORTH is based on using it , is not based on rebuilding it. ABC FORTH could only to be added on a kernel system, that was not mean I built a new FORTH, I was still using FORTH to improve its usage, to make FORTH more convenient in computing aspect only. FORTH original structure had never been changed after added ABC FORTH. Look inside the design of round you can understand this meaning.

My personal testing remarks: Instruction FROUND approved.




















2018年5月1日 星期二

Coefficients of Fourier series


Coefficients of Fourier series

Ching-Tang Tseng
Hamilton, New Zealand
2 May 2018


Artificial music output in my computer was synthesized audio. Off the shelf software package information shows me that sound was mere programmed by combination of sine and cosine functions. It means that all sound waveform of musical instruments could be synthesized by sine or cosine functions. It also means that Fourier series expansion technique could be applied on this application field.

A numerical procedure to find the coefficients of Fourier series from a series of given data was implemented in ABC FORTH. I did this researching. Then I understood how to make a computer generate different music sound. After many different kinds of sound were generated via the specified code from the information manual, I found that synthesized waveform was the main foundation. Other programming factors, for example specifying lengths in time, mixing mode, amplitude, frequency…etc, will mere influence effect of sound output but will not influence the sound style. Piano was still piano, guitar was still guitar, if waveform did not varied. Artificial speech outputs are in the same condition.

To simply the explanation for such an application, this article selected a simple triangle waveform to prepare its data set. Directed Fourier series formulas were used in the program. The coefficients of Fourier series will be calculated out automatically. Data generator program of outcome included as well for going to check. A practical comparative diagram showed the result.


1. Question



Any kind of periodic signal waveform could be represented by its equivalent Fourier series, its general expression are:

f(x) = ½ a(0) + a(1) cos(x) + a(2) cos(2x) + … + a(n) cos(nx)
+ b(1) sin(x) + b(2) sin(2x) + … + b(n) sin(nx) (1)

a(n) = 1/pi ∫ f(x) cos(nx) dx (2)

b(n) = 1/pi ∫ f(x) sin(nx) dx (3)

Integration range is from 0 to 2pi. (2), (3) expressions could be rewritten in terms of summation format:

a(n) = 1/m  f(k) cos ( nk * (2pi / 2m) ) (4)

b(n) = 1/m  f(k) sin ( nk * (2pi / 2m) ) (5)

Summation range is from k = 1 to k = 2m.

According to expressions (1) to (5), assume that k sets data were given, and m = k/2. We are going to find n pairs of coefficients: a(n) and b(n), n = k/2 as well.

Owing to the integration in one cycle = 0, we got a(0) = 0, a(0) will be neglected.

a(0) = 1/pi ∫ f(x) dx = 0
Integration range is from 0 to 2pi.

To simplified our question, we restricted given data are exactly in 2pi periodic range only, and let 12 data points are distributed in equal distance. Data are:


2pi*(1/12) 0.6666
2pi*(2/12) 0.3333
2pi*(3/12) 0.0
2pi*(4/12) -0.3333
2pi*(5/12) -0.6666
2pi*(6/12) -1.0
2pi*(7/12) -0.6666
2pi*(8/12) -0.3333
2pi*(9/12) 0.0
2pi*(10/12) 0.3333
2pi*(11/12) 0.6666
2pi*(12/12) 1.0

The waveform of given data is showing in fig.1.

Fig.1

What are the coefficients of a(1) to a(6), b(1) to b(6)?


2. Code

\ Calculating Fourier coefficients in Lina64 ABC forth

\ ===============================================
\ (1) Data structure declare heap

12 value km
km 2 / value nm

2 integers k n

km array f
km array t
km array u
nm array a
nm array b

5 reals c d q r m
\ ===============================================

\ ===============================================
\ (2) Data input heap

: input
{{ m = i>r ( nm ) }}
{{ f ( 0 ) = 1.0 e 0 }}
{{ f ( 1 ) = 6.666 e -1 }}
{{ f ( 2 ) = 3.333 e -1 }}
{{ f ( 3 ) = 0.0 e 0 }}
{{ f ( 4 ) = -3.333 e -1 }}
{{ f ( 5 ) = -6.666 e -1 }}
{{ f ( 6 ) = -1.0 e 0 }}
{{ f ( 7 ) = -6.666 e -1 }}
{{ f ( 8 ) = -3.333 e -1 }}
{{ f ( 9 ) = 0.0 e 0 }}
{{ f ( 10 ) = 3.333 e -1 }}
{{ f ( 11 ) = 6.666 e -1 }}
{{ f ( 12 ) = 1.0 e 0 }}
;
\ All parts above are changeable setting.
\ ===============================================

\ ===============================================
\ All parts below are fixed.
\ (3) Main code heap
: main
basic
10 run input
20 for n = 1 to nm

30 let { q = 0.0 e 0 }
40 let { r = 0.0 e 0 }

50 for k = 1 to km
60 let { c = i>r ( k ) }
70 let { d = i>r ( n ) }
80 let { t ( k ) = ( f ( k ) * cos ( ( d * c * f(pi) ) / m ) ) / m }
90 let { u ( k ) = ( f ( k ) * sin ( ( d * c * f(pi) ) / m ) ) / m }
100 let { q = q + t ( k ) }
110 let { r = r + u ( k ) }
120 next k

130 let { a ( n ) = q }
140 let { b ( n ) = r }

150 print " format 1================== "
160 print " ( " ; n ; " ):: " , { q , r }

200 print " format 2=================== "
210 print " a ( " ; n ; " ) = " ; { q }
220 print " b ( " ; n ; " ) = " ; { r }

300 print " format 3=================== "
310 run cr ." a ( " n . ." ) = " q f.
320 run cr ." b ( " n . ." ) = " r f. cr cr
400 next n
410 end ;
\ ===============================================

\ ===============================================
\ (4) Manipulation heap

: check
main
basic
20 for k = 0 to 360
30 let { c = i>r ( k ) }
40 let { d = f(2pi) * c / 360.0 e 0 }
50 let { m = a ( 1 ) * cos ( d ) + a ( 3 ) * cos ( 3.0 e 0 * d ) + a ( 5 ) * cos ( 5.0 e 0 * d ) }
60 run cr d f. 5 spaces m f.
100 next k
110 end ;
\ ================================================


3. Outcomes


main

format 1==================
( 1 ):: 8.2929502277 E -1 -2.484 E -17
format 2===================
a ( 1 ) = 8.2929502277 E -1
b ( 1 ) = -2.484 E -17
format 3===================
a ( 1 ) = 0.8292950227
b ( 1 ) = -0.0


format 1==================
( 2 ):: -3.0 E -19 -1.1 E -18
format 2===================
a ( 2 ) = -3.0 E -19
b ( 2 ) = -1.1 E -18
format 3===================
a ( 2 ) = -0.0
b ( 2 ) = -0.0


format 1==================
( 3 ):: 1.1113333333 E -1 -2.3 E -18
format 2===================
a ( 3 ) = 1.1113333333 E -1
b ( 3 ) = -2.3 E -18
format 3===================
a ( 3 ) = 0.1111333333
b ( 3 ) = -0.0


format 1==================
( 4 ):: -1.0 E -19 -2.0 E -19
format 2===================
a ( 4 ) = -1.0 E -19
b ( 4 ) = -2.0 E -19
format 3===================
a ( 4 ) = -0.0
b ( 4 ) = -0.0


format 1==================
( 5 ):: 5.9571643891 E -2 7.37 E -18
format 2===================
a ( 5 ) = 5.9571643891 E -2
b ( 5 ) = 7.37 E -18
format 3===================
a ( 5 ) = 0.0595716438
b ( 5 ) = 0.0


format 1==================
( 6 ):: 0.0 E 0 0.0 E 0
format 2===================
a ( 6 ) = 0.0 E 0
b ( 6 ) = 0.0 E 0
format 3===================
a ( 6 ) = 0.0
b ( 6 ) = 0.0

OK

4. Discussion


Despite to calculate coefficients of Fourier series by numerical method is not a complicated problem, owing to the calculation must cover on full domain of sine and cosine, this question is a very nice evaluation topic for testing of sine and cosine function. I understood this merit. Before I consolidated my program, I had modified many places in my ABC FORTH system to fit the application of this question.

This topic is totally different from the Fast Fourier Transform topic. All analysis are still kept in the time domain, we dont have to consider frequency domain problem. Tedious calculation makes us to do programming. In fact, this example question is an analytical problem. You are able to find the analytical solution from a general Fourier analysis textbook. Analytical solving procedures are tedious as well. Numerical solution is relatively a directed and a simple calculation.

In this program, only one value km is a must to adjust variable. Km must to be an even number. You are able to set out km = 100, then input 100 data, to get 50 pairs of a(n) and b(n). In real world, engineering data are seldom could be represented as a simple mathematical function, especially in speech record signal aspects. This program is dealing with such kinds of input data.

Input data in the question had been arranged in one period of cycle from 0 to 2pi, and had been arranged symmetric to y-axis. Both conditions are considered for simplify coding.

For the sake of linking the using between my FORTH system and Python Plotting system service, I had spent many days redesigned my floating point number output instructions F. and FS.. At the beginning, both instructions will output total digits of floating point number. After this article has been consolidated, both instructions will output 12 digits (include . or - marks) at deferred setting condition. Variable setting of SIGFP can help to adjust output digits. Besides, I adopted ignored treatment for an infinitesimal number, if the number is less than 1.0E-18, or if the number is less than the current setting 12 digits, F. output will be 0.0.

In Python plotting system, a number with decimal point but without E mark inside will be considered as a floating point. This convenience let me easy to design F. instruction in FORTH system. Now, all output data are able to transfer to Python plotting program directly, whatever how many sets. The following program generated 360 pairs of data. According to it, Python plotting program will plot out the overlapping diagram on the original triangle diagram. Fig.2 is showing the achievement of my program.

: check
main
basic
20 for k = 0 to 360
30 let { c = i>r ( k ) }
40 let { d = f(2pi) * c / 360.0 e 0 }
50 let { m = a ( 1 ) * cos ( d ) + a ( 3 ) * cos ( 3.0 e 0 * d ) + a ( 5 ) * cos ( 5.0 e 0 * d ) }
60 run cr d f. 5 spaces m f.
100 next k
110 end ;


Fig.2




5. References



Hwei P. Hsu. Ph. D. Outline of Fourier Analysis, Electrical Engineering department, Wayne State University, 1972

Chih-Fan Chen, I. John Haas, Elements of Control Systems Analysis: Classical and Modern approaches, University of Houston, 1972

Erwin Kreyszig, Advanced Engineering Mathematics, John Wiley and Sons, Inc. New York, 1967

Murray R. Spiegel, Fourier Analysis, McGraw-Hill Book Company, 1974



6. My personal testing remarks



 Instruction “F.” and “FS.” approved.