Solution of a System of Linear Equations for INTELx64

Solution of a System of Linear Equations for INTELx64 A multi core hyper-threaded solution of a system of linear equations for INTELx64 architecture Richa Singhal ABSTRACT. A system of linear equations forms a very fundamental principal of linear algebra with very wide spread applications involving fields such as physics, chemistry and even electronics. With systems growing in complexity and demand of ever increasing precision for results it becomes the need of the hour to have methodologies which can solve a large system of such equations to accuracy with fastest performance. On the other hand as frequency scaling is becoming limiting factor to achieve performance improvement of processors modern architectures are deploying multi core approach with features like hyper threading to meet performance requirements. The paper targets solving a system of linear equations for a multi core INTELx64 architecture with hyper threading using standard LU decomposition methodology. This paper also presents a Forward seek LU decomposition approach which gives better performance by effectively utilizing L1 cache of each processor in the multi core architectu re. The sample uses as input a matrix of 40004000 double precision floating point representation of the system. 1. INTRODUCTION A system of linear equations is a collection of linear equations of same variable. A system of linear equations forms a very fundamental principal of linear algebra with very wide spread applications involving fields such as physics, chemistry and even electronics. With systems growing in complexity and demand of ever increasing precision for results it becomes the need of the hour to have methodologies which can solve a large system of such equations to accuracy with fastest performance. On the other hand as frequency scaling is becoming limiting factor to achieve performance improvement. With increasing clock frequency the power consumption goes up P = C x V2 x F P is power consumption V is voltage F is frequency It was because of this factor only that INTEL had to cancel its Tejas and Jayhawk processors. A newer approach is to deploy multiple cores which are capable to parallel process mutually exclusive tasks of a job to achieve the requisite performance improvement. Hyper threading is another method which makes a single core appears as two by using some additional registers. Having said that it requires that traditional algorithms which are sequential in nature to be reworked and factorized so that they can efficiently utilize the processing power offered by these architectures. This paper aims to provide an implementation for standard LU decomposition method used to solve system of linear equations adopting a forward seek methodology to efficiently solve a system of double precision system of linear equations with 4000 variable set. The proposed solution addresses all aspects of problem solving starting from file I/O to read the input system of equations to actually solving the system to generate required output using multi core techniques. The solution assumes that the input problem has one and only one unique solution possible. 2. CHALLENGES The primary challenge is to rework the sequential LU decomposition method so that the revised framework can be decomposed into a set of independent problems which can be solved independently as far as possible. Then use this LU decomposition output and apply standard techniques of forward and backward substitution each again using multi core techniques to reach the final output. Another challenge associated is cache management. Since a set of 4000 floating point variable will take a memory approximately 32KB of memory and there will 4000 different equations put up together, hence efficiently managing all data in cache becomes a challenge. A forward seek methodology was used in LU decomposition which tries to keep the relevant data at L1 cache before it is required to be processed. It also tries to maximise operations on set of data once it is in cache so that cache misses are minimum. 3. IMPACT With a 40 core INTEXx64 machine with hyper threading the proposed method could achieve an acceleration of ~72X in performance as compared to a standard sequential implementation. 4. STATE OF THE ART The proposed solution uses state of the art programming techniques available for multithreaded architecture. It also uses INTEX ADVANCED VECTOR SET (AVX) intrinsic instruction set to achieve maximum hyper threading. Native POSIX threads were used for the purpose. Efficient disk IO was made possible by mapping input vector file to RAM directly using mmap. 5. PROPOSED SOLUTION A system of linear equations representing CURRENT / VOLTAGE relationship for a set of resistances is defined as [R][I] = [V] Steps to solve this can be illustrated as Decompose [R] into [L] and [U] Solve [L][Z] = [V] for [Z] Solve [U][I] = [Z] for [I] Resistance matrix is modelled as an array 40004000 of double precision floating type elements. The memory address being 16 byte aligned so that RAM access speeds up for read and write operations. FLOAT RES[MATRIX_SIZE*MATRIX_SIZE] __attribute__((aligned(0x1000))); Voltage matrix is modelled as an array 40001 of double precision floating type elements. The memory address being 16 byte aligned so that RAM access speeds up for read and write operations. FLOAT V [MATRIX_SIZE] _attribute__ ((aligned(0x1000))); LU Decomposition To solve the basic model of parallel LU decomposition as suggested above was adopted. Here as we move along the diagonal of the main matrix we calculate the factor values for Lower triangular matrix. Simultaneously each row operation updates elements for upper triangular matrix. Basic routine to do row operation This routine is the innermost level routine which updates the rows which will eventually determine the upper triangular matrix. For each element of row there is one subtraction and one multiplication operation (highlighted). LOOP B designates row major operation, while LOOP A designates column major operation. Basic Algorithm SUB LUDECOM (A, N) DO K = 1, n – 1 DO I = K+1, N Ai, k = Ai, k / Ak, j DO j = K + 1, N Ai, j = Ai, j Ai, k * Ak, j END DO END DO END DO END LUDECOM Each row major operation (LOOP B) iteration can be independently executed on a separate core. This was achieved by using POSIX threads which were non-blocking in nature. Because of mutual exclusion over the set of data MUTEX locks are not required provided we keep the column major operation (LOOP A) sequential. Also for 2 consecutive elements in one row operation 2 subtraction and 2 multiplication operations are done. These 2 operations each are done in single step using Single Instruction Multiple Data instructions (Hyper threading) Multi core Algorithm SUB LUDECOM_BLOCK (A, K, BLOCK_START, BLOCK_END) DO I = BLOCK_START, BLOCK_END Ai, K = Ai, K / AK, K DO j = K + 1, N Ai, j = Ai, j Ai, K * Ak, K END DO END DO END LUDECOM_BLOCK SUB LUDECOM (A, N) DO K = 1, N – 1 BLOCK_SIZE = (N – K) / MAX_THREADS Thread = 0 WHILE (Thread P_THREAD ( LUDECOMPOSITION_BLOCK (A, K, Thread*BLOCK_SIZE, Thread*(BLOCK_SIZE + 1) ) ENDWHILE END DO END LUDECOM Forward substitution Once LU decomposition is done, forward substitution gives matrix [Z]. Here again Single Instruction Multiple Data instructions are used [L][Z] = [V] for [Z] Backward substitution After forward substitution final step of backward substitution gives current matrix [I] [U][I] = [Z] for [I] Here again Single Instruction Multiple Data instructions are used 5. CACHE IMPROVEMENTS On profiling it is observed that the core processing in above solution happens to be LU decomposition. However if we create threads equal in number to available cores the result was improving but not in same proportion to the number of cores. A VALGRIND analysis of cache performance reveals that because of large size of matrix each row operation was suffering a performance hit due to cache misses happening. If we observe above solution it could be observed any jth is processed for (j – 1) columns. So (j – 1) threads are forked for each iteration of column major operation (LOOP A). The data to be processed refers to same memory location but by the time next operation or thread is forked for the same row the corresponding memory data had been pushed out of lower level caches. Thus cache miss happens. To solve this we adopted a forward seek approach wherein we first pre-process a set of columns sequentially thus enabling more operations on a row to be performed in the same thread. Now the data happens to be at lower level cache as we do not have to wait for another thread to process the same row. Multi core Algorithm with forward seek operation SUB LUDECOM_BLOCK_SEEK (A, K, S, BLOCK_START, BLOCK_END) DO I = BLOCK_START, BLOCK_END DO U = 1, S M = K + U -1 Ai, M = Ai, M / AM, j DO j = K + M + 1, N Ai, j = Ai, j Ai, M * AK, M END DO END DO END DO END LUDECOM_BLOCK SUB LUDECOM (A, N) K = 1 WHILE (K //Forward seek DO J = K, K + F_SEEK LU_DECOM_BLOCK_SEEK (A, J, 0, J, J+F_SEEK) END DO //Multi core K = K + F_SEEK DO L = 1, N – 1 BLOCK_SIZE = (N – L) / MAX_THREADS Thread = 0 WHILE (Thread P_THREAD ( LUDECOMPOSITION_BLOCK (A, L, F_SEEK, Thread*BLOCK_SIZE, Thread*(BLOCK_SIZE + 1) ) ENDWHILE END DO END WHILE END LUDECOM CONCLUSION Results For purpose of computation a sample array of double precision floating point matrix of size 40004000 was taken. Performance numbers were generated on an 8 core INTEL architecture machine. TABLE 4.i A programmer that writes implicitly parallel code does not need to worry about task division or process communication, focusing instead in the problem that his or her program is intended to solve. Implicit parallelism generally facilitates the design of parallel programs and therefore results in a substantial improvement of programmer productivity. Many of the constructs necessary to support this also add simplicity or clarity even in the absence of actual parallelism. The example above, of List comprehension in the sin() function, is a useful feature in of itself. By using implicit parallelism, languages effectively have to provide such useful constructs to users simply to support required functionality (a language without a decent for loop, for example, is one few programmers will use). Languages with implicit parallelism reduce the control that the programmer has over the parallel execution of the program, resulting sometimes in less-than-optimal solution The makers of the Oz programming language also note that their early experiments with implicit parallelism showed that implicit parallelism made debugging difficult and object models unnecessarily awkward.[2] A larger issue is that every program has some parallel and some serial logic. Binary I/O, for example, requires support for such serial operations as Write() and Seek(). If implicit parallelism is desired, this creates a new requirement for constructs and keywords to support code that cannot be threaded or distributed. REFERENCES Gottlieb, Allan; Almasi, George S. (1989).Highly parallel computing. Redwood City, Calif.: Benjamin/Cummings.ISBN0-8053-0177-1. S.V. Adve et al. (November 2008).Parallel Computing Research at Illinois: The UPCRC Agenda(PDF). [emailprotected], University of Illinois at Urbana-Champaign. The main techniques for these performance benefits– increased clock frequency and smarter but increasingly complex architectures– are now hitting the so-called power wall. The computer industry has accepted that future performance increases must largely come from increasing the number of processors (or cores) on a die, rather than making a single core go faster. Asanovic et al. Old [conventional wisdom]: Power is free, but transistors are expensive. New [conventional wisdom] is [that] power is expensive, but transistors are free Bunch, James R.;Hopcroft, John(1974), Triangular factorization and inversion by fast matrix multiplication,Mathematics of Computation28: 231–236,doi:10.2307/2005828,ISSN0025-5718. Cormen, Thomas H.;Leiserson, Charles E.;Rivest, Ronald L.;Stein, Clifford(2001),Introduction to Algorithms, MIT Press and McGraw-Hill,ISBN978-0-262-03293-3. Golub, Gene H.;Van Loan, Charles F.(1996),Matrix Computations(3rd ed.), Baltimore: Johns Hopkins,ISBN978-0-8018-5414-9.

War of 1812 :: essays research papers

List and discuss the events leading up to the War of 1812 and the impact it had on American and Great Britain relations, and the American economy. During Jefferson’s second term in office, fighting between Great Britain and France was posed as a threat to American shipping. Napoleon made the decision to exclude British goods from Europe. As a result, Great Britain decided to blockade Europe and prevent ships from entering or leaving the country. A year later, Britain confiscated American cargoes and seized more than a thousand American ships. France had seized about five hundred American ships as well. America’s anger began to focus mainly on the British, even though France was also involved. This was because of the British policy of impressments. The British decided to capture American sailors, or draft them into the British navy. Also, in 1807, the commander of a British warship insisted on boarding and searching the United States naval frigate Chesapeake. When a United States captain refused to allow him onboard, the British killed three Americans and wounded eighteen. As a result, Jefferson convinced Congress to declare an embargo and ban exporting products to other countries. The Embargo Act of 1807 was eventually lifted in 1809 because it stifled American business. A group of young congressmen, known as the war hawks, were angered by the presence of Native Americans in the Indiana territory. Trobule began when General William Henry Harrison persuaded several Native American chiefs to sign away three million acres of tribal land to the United States government. A confederacy of Native Americans began to organize a fight for their homeland against intruding white settlers. This was lead by Shawnee Chief Tecumseh. Tecumseh’s brother lead the Shawnee in an attack on Harrison, but was defeated. When the war hawks found out that the confederacy was using arms from British Canada, they again called for war. James Madison, another Virginia Republican, declared war against Britain in 1812. He believed that Britain was trying to strangle American trade and weaken the American economy.

Anderlini and Clover Essay

In their article, Anderlini and Clover (2009) speak about China’s and Russia’s desire to purchase IMF bonds. While China considers buying about $50bn of IMF bonds, Russia seeks to spend no more than $10bn for these purposes. Both countries will use these investments according to essential criteria of reasonable returns and safety, which are in no way associated with the countries’ search for additional political power in international contexts. It appears that for Russia and China to purchase IMF bonds means to express their desire to trace and monitor the distribution of international monetary commitments. The money Russia and China are prepared to pay for IMF bonds is expected to help developing countries tackle with the major economic challenges. For example, Russia proposes that IMF uses additional funds to help Ukraine resolve its gas issues with Russia (Aderlini & Clover, 2009). Although the IMF is not very optimistic with regard to sponsoring Ukraine in its balance payment issues, purchasing bonds may shape a good ground for better stability in broader financial markets. Response The fact of Russia and China seeking to purchase IMF bonds signifies the growing international commitment to reducing trade barriers. With the growing realization of the benefits which the reduction of trade barriers can bring internationally, the IMF bonds and additional funds can be readily used to support developing countries in their striving to better trade liberalization and business openness. On the one hand, the developed countries’ desire to stimulate international trade signifies their preparedness to better dialogue with developing countries in terms of business and trade. On the other hand, such openness also provides developing countries with better chances to become a part of the developed business community. As a result, whether the changes in the structure of international financial assets help reduce trade barriers also depends on how well countries and organizations manage them. Response 1 In his article, Bogoslaw (2009) suggests that the time has come when India, Brazil, and China should become the major investment targets. Given that the state of economy is not limited to economic markets in the U.  S. , it is more than important to look beyond the boundaries of the American economic attractiveness and to provide other countries with a better chance for economic growth. It should be noted, that the concept of market economic system is integrally linked to the concept of economic freedom, and where countries seek to implement the principles of market economy these imply the absence or minimization of governmental involvement. In case of China, India, and Brazil, governments still remain the powerful elements of economic growth. Simultaneously, dozens of smaller developing countries need additional investments for their gradual transition to free market relationships. Thus, not Brazil or India with their well-established economic images, but other developing countries with sound legal systems and investment opportunities should attract additional funds. In any case, stocks and investments always involve risk, and if investors believe that by cooperating with India or China they secure themselves from the major losses, they are deeply mistaken. Response 2  For many years, embargos have been an effective measure of economic and diplomatic discipline. The leading world powers frequently apply to embargo as the measure of last resort, and whenever countries are unwilling to follow the basic principles of international legal or economic conduct, embargos appears the most reliable method of imposing balanced legal and economic requirements on them. It appears that to stop supplying countries with the critical resources is more important that trying to persuade such countries to change their convictions and political beliefs. It should be noted, that embargo implies putting a legal ban on commerce, and individuals are those who suffer these limitations the most. As a result, whether embargo is an effective measure depends on what perspective one chooses to review its benefits and drawbacks, but that embargos significantly reduce the scope of the major business operations and prevent individuals from achieving their individual purposes is clear.

The 1920s - Research - 2133 Words

The Roaring Twenties, the Jazz Age, the Golden Age; what happened in this decade that made it so roaring, jazzy, and golden? What made up the twenties? Known for fun, style, and prosperity, the ‘20s were one of the most exciting, controversial, and productive periods in America. This paper will cover some (not all) of the significant events and inventions that happened in this revolutionary decade. Well-known parts of the Jazz Age include, jazz, flappers, fashion, and the radio. Also notorious for being a reckless, irresponsible, and materialistic era, the 1920s also had some infamous problems; Prohibition, gangsters, and the start of the great Depression. Many new things arose in this era. The new technologies that became†¦show more content†¦The first Oscar movie was made by Paramount pictures. Metro Goldwyn Mayer film-making studios was also founded. Mickey Mouse became Americas favorite cartoon character in Steamboat Willie. Pooh Bear was also created and was very p opular for young children. The music of the time was jazz . . . Jazz, blues, and ragtime. These were all popular along with swing dancing. The Charleston was one of the better-known dances. Most of the twenties forms of entertainment didnt die out, but evolved with time into something more advanced. Movies, radio, and the music all survived (not so much the specific styles of music). But the roots of these kinds of entertainment can all be found in that decade. Art and theater were more popular than ever in the 1920s. Early modernism in art began at the turn of the century and continued through World War II. Modern styles of art included abstract expressionism, realism, and surrealism. The best museums featured shows by the important artists who used these styles. Broadway reached an all time peak. There were 276 plays offered in 1927 in New York City. (This is a lot compared to only 50-something in the 1970s.) Historians argue over exactly how many theaters there were. Some say eig hty, some say seventy, but everyone agreed that Broadway was booming in the 1920s. After the war, the American population was moving more and more into the cities. In response to the many social changes in America, the newShow MoreRelated1920s Fashion - Research Paper1069 Words   |  5 PagesThe 1920’s fashion was a period of liberation, change, and even more importantly a movement towards the modern era. Fashion in the 1920’s varied throughout the decade but one could see the noticeable change from the previous fashion statements and eras. At the start of the decade, women began emancipating themselves from the constricting fashions by wearing more comfortable apparel. As women gained more rights and World War I forced them to become more independent, flappers came to be, mass-producedRead MoreAnalysis Of The Movie 1920 American Film 1318 Words   |  6 Pages1920’s American Film During the 1920s, American Film was at the peak of its glory. 1920s Film was the biggest form of entertainment and a weekly pastime for millions of Americans, regardless of race and social background. Silent films continued to improve and innovate the film industry. Hollywood established themselves as an American force and produced hundreds of silent films. Also, Hollywood became the birthplace of â€Å"movie stars† such as Janet Gaynor, Rudolph Valentino, and Charlie Chaplin. MovieRead MoreWatson and Rayner’s Classical Study with Llittle Albert Essay1726 Words   |  7 PagesIn the following essay I will be looking into the study conducted by Watson and Rayner (1920) on a small child known as ‘Little Albert’. The experiment was an adaptation of earlier studies on classical conditioning of stimulus response, one most common by Ivan Pavlov, depicting the conditioning of stimulus response in dogs. Watson and Rayner aimed to teach Albert to become fearful of a placid white rat, via the use of stimulus associations, testing Pavlov’s earlier theory of classical conditioningRead MoreEssay on Economic Expansions in 1920s1452 Words   |  6 PagesDuring the 1920s, there was a rise in economy of the United States. The people of the United States and its territories enjoyed a prosperous life, as the economy grew 7 percent per year between 1922 and 1927. In this period, also referred as â€Å"Roaring Twenties†, there was high economic growth with increase in the living standards of Americans. According to the textbook, â€Å"Nation of Nations†, the reasons for the economic expansions in the nineteenth century were due to the boom in the industrial sectorRead MoreEssay on Book Review: Daily Life in the United States, 1920-19401194 Words   |  5 PagesBook Review: Daily Life in the United States, 1920-1940 The way Americans lived their lives was drastically changed between the years of 1920 and 1940. Many different events and advances in technology happened within the country during this time period. Events such as the stock market crash in 1929, the dust bowl of the 1930’s, and, due to an increase in urbanization, the uprising of major cities. Also advances in technology transpired, such as the invention of the radio and Henry Ford’s assemblyRead MoreGender Roles In The Great Gatsby1736 Words   |  7 PagesThe Great Gatsby by F. Scott Fitzgerald, demonstrates the expectations of women and their relationships to men in 1920’s New York City through one of the main characters, Daisy Buchanan. A vast majority of Daisy’s actions are to entice and cater to the superior men of the novel. Through this, I was able to reflect upon the evolution of society’s stereotypes surrounding women from the 1920’s. Initially, from reading the novel, I learned a bout the period of the roaring twenties and how the aspect ofRead MoreThe Discovery Of Insulin And Penicillin And The Development Of The U.s. Health Care System1745 Words   |  7 PagesWhen most people think of America in the 1920s, the thing that comes to mind is the phrase ‘Roaring Twenties’ accompanied by the thought of flappers and Gatsby. In fact, if a person were to Google images of ‘the Roaring Twenties,’ there is a very little variation in results. Many people do not know that the 1920s was more than an age of economic prosperity and defying prohibition; it was also a time of great advances in health care and medicine in the United States. The discovery of insulin and penicillinRead MoreAshford 4: - Week 3 - Assignment1335 Words   |  6 PagesFinal Paper Preparation This assignment will prepare you for the Final Paper by initiating the research process and helping you map out specific events and developments which you will explore in depth in your paper. Review the instructions for the Final Paper laid out in Week Five of the online course or the Components of Course Evaluation section of the Course Guide before beginning this project. Note, that for the Final Paper you will need to discuss at least six specific events or developmentsRead MoreEssay about Roaring Twenties772 Words   |  4 Pages The Roaring Twenties The decade of 1920-1929 was a time of great change, reform, improvement, adjustment and alteration of everything Americans had come to rely on. In other words everything changed. Not one part of common life was unaffected. Exciting new events happened in sports, entertainment, science, politics, communication and transportation. It was the age of prohibition, it was the age of prosperity, and it was the age of downfall. The twenties were the age of everything. It has beenRead MoreRoaring Twenties Essay839 Words   |  4 PagesThe Roaring Twenties The decade of 1920-1929 was a time of great change, reform, improvement, adjustment and alteration of everything Americans had come to rely on. In other words everything changed. Not one part of common life was unaffected. Exciting new events happened in sports, entertainment, science, politics, communication and transportation. It was the age of prohibition, it was the age of prosperity, and it was the age of downfall. The twenties were the age of everything. It has