Computerized evaluation of digital orthopedic Moiré pictures

6th ElectricalEngineering and Mechatronics InternationalConference

11 October 2013,Debrecen,Hungary

,

COMPUTERIZED EVALUATION OF DIGITAL ORTHOPEDIC MOIRE PICTURES

Petra BAllA, GÁbor Manhertz, Ákos Antal Ph.D.

Budapest University of Technology and Economics,Budapest,Hungary Department of Mechatronics, Optics and Mechanical Engineering Informatics

petra.balla@mogi.bme.hu

manhertz@mogi.bme.hu

antal.akos@mogi.bme.hu

KEYWORDS: *Moire method, LabVIEW algorithm, image evaluation, orthopaedics*

Abstract - By the superimposition of two similar periodic structures translated, or rotated relative to each other, a new structure arises with longer periods. This one is called Moire pattern and it consists of the Moire fringes. Moire imaging (used mostly in spinal deformities) is non-contact, fast, and its harmful radiation does not load patients and can be repeated any number of times. For detecting the spinals shape, a software has been implemented in LabVIEW programming environment with several image processing method. The shape of the spine could be sufficiently reproduced using this program. Screening can be established without X-Ray images, which is much faster and healthier for patients.

1. INTRODUCTION

The digital, computer-aid Moire device is being tested for a few years. It is used especially to follow patients' conditions during the treatment, but it can be used also for screening healthy patients. For screening it is necessary to get an algorithm to evaluate the Moire pictures. Originally this method would have been a substitution instead of manual examination. After a suitable algorithm is going to be discovered, this method can replace the X-Ray technique.

The patient is not to be touched during the examination, and the number of examinations of the patient's

back could be increased using this method. This technique is fast and quite cheap, so it can be performed any number of times.

2. THE MOIRE EFFECT

Optical measurement methods of three-dimensional surface metrology became rapid and effective tools

especially in the field of orthopedics [1-5][7][10-12], where moire imaging (used mostly in spinal deformities) is considered to be capable of measuring and graphically displaying whole surfaces instantaneously. [13]

By the superimposition of two similar periodic structures translated, or rotated relative to each other, a new structure arises with longer periods. This one is called moire pattern and it consists of the moire fringes.

Simple moire patterns can be observed when superposing two transparent layers compnsmg periodically repeating opaque parallel lines as shown in *Fig. 1. *The lines of one layer are parallel to the lines of the second layer. [22]

Fig. 1: The Moire method [23]

The superposition image does not change if transparent layers with their opaque patterns are inverted. One of the layers is denoted as the base layer and the other one as the revealing layer *(Fig. 1).*

The superposition image of *Fig. 1 *outlines periodically repeating dark parallel bands, called moire lines.

Spacing between the moire lines is much larger than the periodicity of lines in the layers.

Light areas of the superposition image correspond to the zones where the lines of both layers overlap. The dark areas of the superposition image forming the moire lines correspond to the zones where the lines of the two layers interleave, hiding the white background. The labels of *Fig. 2 *show the passages from light zones with overlapping layer lines to dark zones with interleaving layer lines. The light and dark zones are periodically interchanging.

Fig. 2: Superposition of two parallel/ayers

2.1 Using the Moire method in orthopedics

During the Moire surface measurement a grating (or its shadow) is projected to the surface of the model, where this grating will take a distorted form due to the model's shape. Another grating (which can be the same) is located between the model and the observer (that can be for example a camera or a person's eye). As the second grating remains unchanged, it can be considered as a reference grating. The axis of the observer encloses a certain angle with the axis of the projection by intersecting each other on the surface. [14-16]

Several points can be found on the Moire pattern what have the same distance from the reference surface (or base plane). By connecting these points, contours can be created, just like at the topographical maps, in this way the test surface can be described with these lines.

By superimposing two periodic or quasi-periodic structures, the Moire phenomenon can be observed. When the two structures have the same or slightly different line spacing and their lines are set approximately parallel, a new coarse pattern appears. This pattern is known as the Moire fringe pattern. The spacing and orientation of the Moire fringes depend on the spacing and orientation of the structures being overlapped whereas the visibility of fringes is related to the width of transparent or black lines with respect to the line spacing of the structures. Moire topographical methods can be distinguished as: the basic grating-shadow, the grating-projection, the grating-TV and the synthetic, computer generated grating methods. Shadow Moire is a contour mapping technique that involves positioning a grating close to an object and observing its shadow on the object through the grating. Thus, the basic grating-shadow method offers the best accuracy and the simplest arrangement because the projected grating and the master grating are identical, so they have the highest degree of binding. The disadvantage of the shadow Moire technique is that the master grating must have similar size as the measured object.

The Moire method has several advantages, such as being fast, relatively cheap, highly sensitive, and has graphical results. It is also essential that the method offers a non-contact surface measurement 1201 which

does not burden the patient with radiation, unlike X-ray analysis. Moire measurement can be a supplementary method for determining the necessary further examinations and treatments of patients. The Moire fringes formed on the human back allows the examiner to find the characteristic points of the spine easily.

3. THE MOIRE EQUIPMENT

This equipment is designed and developed with the help of orthopedic medicals. Actually it is classical projection Moire equipment 117 191, but video projector is used instead of the traditional projector. A picture is taken from the reference area first, and then the patient's back appears in the examined area. The pattern and the settings are left without change. Because of the examined object the pattern will be strained.

Another photo is taken of this, and the computer £191 produce a Moire picture from these data. 1231 The image can be evaluated with one of the previously described algorithms. In this picture the contours are visible, subjective evaluation also can be applied.121 231

4. SPINAL DEFORMITY TYPES

Scoliosis is a weed deformity on the spine, mostly in the frontal plane. It has two different types, functional and structural scoliosis. In case of functional scoliosis there isn't any rotation. Rib deformations cannot be observed. Angle of that deformity will stagnate without a treatment, but physiotherapy can be the suitable method to cure it.By structural scoliosis there are weed lateral deformities on the spine with torsion. Rib or chest deformations can be observed because of vertebrae torsion. Patients have to wear brace until the end of ossification, without this the disease will deteriorate. Young girls are more often concerned compared to boys. It could be discovered in rapid growth phase of the bones, mostly at teenagers. Sadly there are 3-4 years old little children with large deformities who have to wear rigid brace until the end of ossification, as the best treatment of the problem. Scoliosis without treatment can cause pain, aesthetic problems, disability and even failures in blood circulation.

Fig.3: Scoliosis on X-Ray and on Moire pictures f&J

The Cobb angle is measured on plane radiographs by drawing a line through the superior endplate of the superior end vertebra of a scoliotic curve, and another line through the inferior endplate of the inferior most vertebra of the same scoliotic curve, and then measuring the angle between these lines.[ 6][20]

Fig. 4: Cobb angle[25]

The second type of spine deformity is the Scheuermann disease. At this disease the spine is straight in the frontal plane, but the kyphosis is much rounder than at healthy people. Nowadays the lumbar kyphosis is also very frequent.

Fig. 5: Scheuermann disease [9]

5. DETECTING THE SPINAL'S SHAPE WITH A SOFTWARE ALGORITHM

For detecting the spinal's shape, a software has been implemented in LabVIEW programming environment with the Vision & Motion Toolkit. LabVIEW was chosen because it is a graphical (higher-level) programming method, and the available palettes simplify the implementation of the various functions and provide greater traceability. Additionally, the Vision Toolkit can create several layers on a picture during the image manipulation and several help point and area can be defined as well. These objects are called in LabVIEW as Region of Interest (ROI).

To reach the objective a preliminary plan was drawn up, which looks like the following. First an appropriate mesh is created on the Moire picture. This mesh generates intersection points with the Moire fringes. These points are used to calculate an nth-order fitting curve. After generating this curve two methods are used to detect a point on the spinal curve. The first calculates the inflection points, the second one the minimum points of the fitted curve. Because of the spine has to be located in a valley on the back, to proceed the process, two points (2 inflection) need to be selected by this valley. After this, the program generates two tangential lines based on the selected points. The intersection of these two lines will give a point of the perpendicular surface that consist a point of the spinal curve. The minimum point method works very simply. The program fmds the minimum point of the generated fitting curve, and that point will be one a point of the spinal curve. The final shape of the curve can be obtained by executing this algorithm on every horizontal line of the mesh which was created at the beginning of the method. At the end, a curve fitting method is needed to be used to get the shape of the spine.

The following sections will explain each step of the algorithm.

5.l The preparation of the Moire picture

At the beginning of the process a Moire picture needs to be loaded in. After selecting the file, which extension can be ".jpg", ".png", ".bmp", an indicator will show the loaded image.It has to be noted that for the future researches, the program can load digital X-Ray files with ".dicom" extension too. After loading the picture, an area needs to be selected which contains the necessary information for detecting the spinal's curve-the Moire fringes of the back. This area needs to be selected manually with a ROI rectangle

(or any other polygon).

After the selection a cutting method is applied on the picture.

Fig. 3: Cutting that part of the picture which contains the Moire fringes of the back (green lines- ROI rectangle)

5.2 Mesh generation

After cutting the necessary area of the picture, the program needs to generate a mesh on it. This mesh will help to obtain points which will be the first base step in determining the curve of the spinal.

The program creates a layer where the image processing methods will be executed. First, on this layer, a mesh will be created. The creation is based on the horizontal and vertical resolution of the picture and on a grid size constant. The desired size (in pixels) of a grid can be set by this constant. The smaller the grid size is, the more accurate further calculations can be performed. The program creates two-two arrays which contain the start and the end points of every horizontal and vertical line. These points were generated with two "for loops": one for the horizontal and one for the vertical lines. The number of iterations of a loop depends on the grid size constant (how many lines needs to be generated). A start and an end point will be determined in every iteration with adding the grid size constant to a shifted width value (which was calculated in the previous iteration). The importance of the grid size will be shown in the next chapter.

Fig. 4: a.) Loose mesh b.) Dense mesh

(red lines - the horizontal and vertical lines of the generated mesh)

5.3 **Acquiring intersection points with the Moire fringes**

The mesh created in the previous step is used for generating intersection points. These points are derived from the intersection of a horizontal mesh line with the Moire fringes. One opportunity for further development is that the program searches all intersection points on a horizontal line, because now the points are selected manually.

The manual selection functions are the followings. First, the user needs to observe visually the Moire picture and the created mesh, looking for possible intersections. After than a mouse click needs to be executed near it. The program checks the coordinates of the mouse click on the picture and selects the closest horizontal and vertical mesh line crossing point (all of these points are collected into an array). That's why the grid size is so important. When the grid size is large, the result can be inaccurate. Because of the small grid size, the mesh will be denser, more horizontal and vertical line crossing points will appear, and the approximation will be better. Namely, the result intersection point could be better fitted to the analyzed Moire fringe. The approximation algorithm calculates an array filled by sums. These sums originates from the following equation:

XM, YM- the coordinates resulted by the mouse clicking

Xi, Yi - the ith coordinate pair from the array that contains the horizontal and vertical mesh lines crossing points

C - adjustable constant, which can improve the approximation

S - the result sum

The algorithm calculates these sums by using every mesh line crossing point, and then the lowest result will be selected. The array index of this element is equal to the closest crossing point's index in its storage array. When the calculation is over the intersection a point will be highlighted on the picture *(Fig. *5).

Fig.5: Intersection points of a selected horizontal line and the Moire fringes

(black points - the generated intersection points)

5.3.1 Weighting of the points

After generating the intersection points, these points need to be weighted. This weight describes which Moire fringe belongs to a point. The weight is derived from the order of the Moire fringe which order is important because of the followings. There is a relationship between the position of the spine, the symmetry of the backs plain and its surface features, so the geometry of the scapula and its environment carries information about the spines elevation position. This information is contained by the ordered Moire fringes. The weighting uses these orders in a way that the value of the order is added to the corresponding y-coordinate of the intersection point.

Fig. 6: The orders of the Moire fringes on the back

(yellow- dh order, green- rt order, blue- 2nd order, red- 3rd order)

5.4 Calculating an nth-order fitting curve

In the next phase an nth-order polynomial curve is fitted to the selected and weighted intersection points. In practice, usually a maximum 5th-order polynomial is used to accomplish the fit, but several times during this research a multi-degree curve was calculated as well. The LabVIEW development system offers several built-in fitting processes, that's why is so easy to perform this step. The built-in, general polynomial fitting method uses every time Cholesky-algorithm during the image processing. The more in-depth description of the fitting algorithm is beyond of the scope of the paper.

5.5 The two methods for determining a point of the spinal's curve

The main part of the research is the determination methods of a point of the spinal's curve. To perform this, two methods had been implemented. The first one calculates the inflection points, the second one the minimum points of the curve fitted during the previous step. Because of the spine has to be located in a valley on the back, the analysis needs to be performed by this. With the further processing of these characteristic points, a point can be generated on a surface which is perpendicular to the current image processing plane. The fmal shape of the spinal will be located on this plane and fitted to the generated point. The following two sections will represent the two methods for calculating this point.

5.5.1 Calculation based on inflection points of the fitted curve

The first method is based on the fitted curves inflection points. To locate the inflection points of a function, it needs to be differentiable twice.

To find out whether there are inflections in the curve, the second derivative needs to be checked. If there is a

solution to zero, then there is an inflection at the resulted abscissa value. To obtain the inflection point correctly the ordinate value needs to be calculated after substitution.

Fig. 7: Finding the inflection points of the nth-order polynomial curve

(red- the polynomial curve, green -inflection points)

In the next step those two inflection points need to be selected which are located by the valley of the back. In these points the algorithm creates one-one tangential lines which lines must intersect each other. This resulted intersection point will be used in the next step to determine the spinal's shape.

Fig. 8: Determining the intersection point of the two tangential lines

(red- the polynomial curve, green -the inflection points, blue lines -the tangential lines, red point- intersection point)

5.5.1 Calculation based on minimum point of the fitted curve

The second method is based on the fitted curve's local minimum point in the specified area. To locate the local minimum point of a function, it needs to be differentiable twice.

To fmd out whether there are extremes, the first derivative of the curve needs to be checked. If there is a solution to zero, then there is an extreme at the resulted abscissa value. To defme the nature of this extreme, the second derivative needs to be calculated in that abscissa value. This point is a local minimum if the calculated second derivative value is greater, than zero. To obtain the minimum point correctly the ordinate value needs to be calculated after substitution. This minimum point will help in the next phase to determine the spinal's shape.

Fig. 9: Finding the inflection points of the nth-order polynomial curve

(red- the polynomial curve, green - minimum point)

5.6 The determination of the spinal's shape

After executing the steps interpreted in the previous sections on every horizontal mesh line, an array of points will be generated. These points are the points of the spinal curve located on a surface which is perpendicular to the surface where the Moire picture processing was done.

To obtain the shape of the spinal's curve, a fitting method needs to be used again. Therefore, the algorithm

operates here too with the built-in, general polynomial fitting method (like at the nth-order polynomial curve fitting phase). It needs to be mentioned that the order of the polynomial here should be only 3rd or 4th for better curve approximation.With this method a curve can be generated, as it's shown on Fig. 10.

Fig. 10: The determined shape of the spinal

(red- the spinal's shape resulted by a fitting method

blue -the points which had been calculated in the previous phase)

6. CONCLUSION AND FURTHER DEVELOPMENT OPTIONS

This new algorithm can be a suitable method to evaluate Moire pictures. In one case the shape of the spine was correctly discovered by the algorithm. This curve contains only the main line of the spine, without the borders of the vertebra. The algorithm needs to be checked in more cases, and it is very important to ensure the classification.

There are more types of spinal deformities, and in scoliosis there are also more subtypes, e.g. 1, 2, or 3,

abnormal curve(s) can be located on the spine. All subtypes have to be evaluated by the algorithm to verify the compliance of the method. 20-30 pictures have to be separated in all groups and after the evaluation the results have to be compared to the Cobb angles of X-Ray images in order to check for errors.

The Scheuermann disease is a different type of deformity, because of that a new algorithm needs to be developed to evaluate pictures taken from patients who have Scheuermann disease. Probably the higher density of the patterns can be a solution.

The automation of the program can help the faster evaluation, so it is important to develop it in the future.

This result can be enough to establish the right diagnosis, so it is a huge help for screening healthy people. Unfortunately, not so many curves had been created as the result of the program, so rib deformations cannot be observed on the spine model. This problem can be solved by complete screening with physical examination.

The development of the program can offer us more opportunities of using. A wide zone could be examined next to the spine on the pictures to get information about the ribs condition and location.

If a model of the spine (with all vertebrae) would be used, the curve can be fitted into the model, and after evaluation a real spine, its shape can be generated. Using this spine-model the Cobb angle, the vertex of the curvature and the rotation can be got also measured.

The final application of the algorithm can be the substitution of the X-Ray technique, because the screening

of the patients would be healthier, faster and much cheaper for orthopaedics.

REFERENCES

1. Theocaris P. S.: Moire Fringe in Strain Analysis, Pergamon Press (1969).

2. Patorski K., Kujawinska M.: Handbook of the Moire Fringe Technique, Elsevier, Amsterdam (1993).

3. Durelli A. J., Parks V. J.: Moire Analysis of Strain, Prentice-Hall, Englewood Cliffs, New Jersey, (1970).

4. Parks V. J.: Geometric Moire, SEM Handbook of Experimental Mechanics, Prentice-Hall, Englewood Cliffs, New

Jersey, (1987).

5. Parks V. J.: Strain measurement using grids, Optical Engineering, Vol. 21; (1982): 633-639.

6. Orosz M.: Conservative treatment ofthe idiophatic scoliosis, Pediatrics (2005); 56(6): 651-657.

7. Adir, I V., Wljk, M C., Armstong, W: D.: Moire topography in scoliosis screening. Clin. Orthop., Clin. Orthop., (1977); 129: 165-171.

8. *Petra Balla, Ferenc Marlok, Katalin Prommer, Szilvia Nagy, Orsolya Schulcz, Gyula Kocsis, Benjamin Magocsi*

Processing of Digital Moire Images in Salus Ortopedtechnika Kft., V. Magyar Biomechanika Konferencia, 2013. mAjus 24.-25, Budapest

10. M. Batouche, R. Benlamri and M K. Kholladi: A computer vision system for diagnosing scoliosis using moire images, Computers in Biology and Medicine, Volume 26; Issue 4; (1996): 339-353.

11. Ruggerone M., Austin J. H M.: Moire Topography in Scoliosis, Correlations with Vertebral Lateral Curvature as

Determined by Radiography, Physical Therapy, Vol. 66; No.7; (1986): 1072-1077.

12. Laulund, T., Sejbjerg, J. 0., Herlyck, E.: Moire Topography in School Screening for Structural Scoliosis, Acta

Orthopaedica, Vol.53; No.5; (1982): 765-768.

13.T. Laulund, J. 0. St jbjerg and E. Ht rlyck: Moire Topography in School Screening for Structural Scoliosis, Acta

Orthopaedica, (1982); Vol. 53; No.5.: 765-768.

14. Zselyke Gaal, Akos Antal, Tamas Peter: Scoliosis testing features on the basis of electronically generated Moire patterns, Biomechanica Hungarica (2010):ill:(1): 84-92.

15. Gaal Zselyke, Antal Akos, Tamas Peter: Statiscical Analysis of Determining the Rate of Scoliosis from

Electronically Generated Moire Patterns, In: Proceedings of Seventh Conference on Mechanical Engineering. Budapest; Hungary; (2010.05.25-2010.05.26) Budapest: 796-803.

16. Gail/ Zselyke, Antal Akos, Tamils Peter: Scoliosis Testing Features on the Basis of Electronically Generated Moire Patterns, In: 8th IEEE International Symposium on Applied Machine Intelligence and Informatics. Herlany, Slovakia, (2010.01.28-2010.01.30): 335-340.

17.Takasaki H.: Moire Topgraphy, Applied Optics, (1970); Vol. 9.; No.6.: 1467-1472.

18.Takasaki H.: Moire Topgraphy, Applied Optics, (1973); Vol. 12.; No.4.: 845-850.

19. Wenzel Klilra, Antal Alws, Molnar Jozsef, Toth Bertalan, Tamils Peter: New Optical Equipment in 3D Surface

Measuring, Journal of Automation Mobile Robotics & Intelligent Systems 3: (4):29-32.(2009)

20. Neugebauer H.: The different methods of measuring the curvature of scoliotic spine In. Moire Fringe Topography and Spinal Deformity, Proceedings ofthe 2nd. International Symposium, Gustav Fischer Verlag; Stuttgart (1983): 17-26.

21.Frontino G, Negri V, Rinaldi E.: Photographic moire method for preliminary diagnosis of spine deformity, Ateneo

Parmense Acta Biomed. (1980); 51(1):33-8.

22. Hyoung Seop Kim, Kazufumi Ishida, Seiji Ishikawa, Yoshinori Ohtsuka, Hisashi Shimizu: Discriminating spinal deformity employing centroids difference on the moire images, Systems and Computers in Japan, Vol. 32; Issue 7:20-28.

23. Gaal Zselyke, Tamils Peter, Antal Akos: Calibration and Measurement Test of Medical Moire Equipment, In: XXN. microCAD International Scientific Conference. Misko1c, Hungary, (2010.03.18-2010.03.20) Misko1c: Miskolci Egyetem Innovaci6s es Technol6gia Transzfer Centruma; ISBN:978-963-661-919-0):41-46.

24. The Basic of Line Moire Patterns and Optical Speedup, Abstract, Emin Gabrielyan Switzemet Sari, Scientific

Park of Swiss Federal Institute of Technology, Lausanne (EPFL)