Parallel & Hybrid Programming

UNIVERSITY OF BURGOS Area of Electronic Technology Parallel & Hybrid Programming. José María Cámara Nebreda, César Represa Pérez, Pedro Luis Sánchez Ortega Parallel & Hybrid Programming . 2015 Area of Electronic Technology Elect romechanical Engineering Department University of Burgos Introduction ............................................................................................................................................. 5 Activity 1: MPI Matrix Multiplication ...................................................................................................... 7 OBJETIVES ............................................................................................................................................ 7 THEORETICAL CONCEPTS..................................................................................................................... 7 PRACTICAL EXERCISE ........................................................................................................................... 7 QUESTIONS .......................................................................................................................................... 8 Activity 2: Performance Assessment ....................................................................................................... 9 OBJECTIVES .......................................................................................................................................... 9 THEORETICAL CONCEPTS..................................................................................................................... 9 PRACTICAL EXERCISE ......................................................................................................................... 12 QUESTIONS ........................................................................................................................................ 13 Activity 3: Introduction to Hybrid Programming ................................................................................... 14 OBJETIVES .......................................................................................................................................... 14 THEORETICAL CONCEPTS................................................................................................................... 14 PRACTICAL EXERCISE ......................................................................................................................... 17 QUESTIONS ........................................................................................................................................ 17 Activity 4: Hybrid Programming ............................................................................................................ 18 OBJECTIVES ........................................................................................................................................ 18 THEORETICAL CONCEPTS................................................................................................................... 18 PRACTICAL EXERCISE ......................................................................................................................... 19 Activity 5: MPI vs OpenMP .................................................................................................................... 20 OBJECTIVES ........................................................................................................................................ 20 THEORETICAL CONCEPTS................................................................................................................... 20 PRACTICAL EXERCISE ......................................................................................................................... 20 Activity 6: Submitting jobs to a cluster.................................................................................................. 21 OBJETIVES .......................................................................................................................................... 21 THEORETICAL CONCEPTS................................................................................................................... 21 PRACTICAL EXERCISE ......................................................................................................................... 26 Activity 7: Job scheduling ...................................................................................................................... 27 OBJETIVES .......................................................................................................................................... 27 THEORETICAL CONCEPTS................................................................................................................... 27 PRACTICAL EXERCISE ......................................................................................................................... 30 Activity 8: Job scheduling II ................................................................................................................... 31 OBJETIVES .......................................................................................................................................... 31 THEORETICAL CONCEPTS................................................................................................................... 31 PRACTICAL EXERCISE ......................................................................................................................... 31 Activity 9: Job scheduling III .................................................................................................................. 32 OBJETIVES .......................................................................................................................................... 32 THEORETICAL CONCEPTS................................................................................................................... 32 PRACTICAL EXERCISE ......................................................................................................................... 32 Activity 10: Performance competition. ................................................................................................. 33 OBJETIVES .......................................................................................................................................... 33 THEORETICAL CONCEPTS................................................................................................................... 33 PRACTICAL EXERCISE ......................................................................................................................... 33 Appendix A: Installing DeinoMPI ........................................................................................................... 34 Installation ......................................................................................................................................... 34 Configuration ..................................................................................................................................... 34 Launching Jobs .................................................................................................................................. 34 Graphic Environment ........................................................................................................................ 34 Deino MPI manual. Available at: http://mpi.deino.net/manual.htm ................................................... 38 Appendix B: Project Configuration in Visual Studio 2010 ..................................................................... 39 Appendix C: Configuration of MS-MPI. ................................................................................................. 44 LABORATORY GUIDE Introduction 5 Introduction Our interest will be focused on parallel programing for multicomputer MIMD machines. Our application programs will split into several processes and each one will have the potential capability to be executed on a different node of our cluster. The processes created by the user will cooperate to achieve a common computational objective. The collaboration will be possible due to communication and synchronization tools provided by the programming environment. Communication is implemented in the form of message exchanging. Most of the scenarios proposed admit a number of different parallel solutions. We should try to come up with the most advantageous in terms of system performance. To do so we must take into account: • We will try to increase performance (execution time). To do so, we will try to squish the application’s potential locality, that is, its capability to work with local data avoiding the need for much information exchange between processes. • Another important point is “scalability”. In a hardware environment, where the amount of available resources is unknown at programming time, the application must scale to make the most of the available resources at any time. Parallel programing is not an easy job. The theory around the development of concurrent and parallel software is beyond the scope of this course but, we will provide some hints. Parallel programming, as well as sequential programming is a creative task; what is about to be exposed is nothing more than a series of steps we recommend to follow when facing a parallelization. Let’s split up the process in 4 steps: • Fragmentation: this initial step is meant to find potential parallel structures within the problem to be solved. As a first approach, we may try to decompose the job in as many small parallel tasks as possible. Two criteria can be followed to carry out this decomposition: o The functional way: seeks for possible divisions in the job to be carried out by paying attention to its nature. o The data way: pays attention to the nature of the data to be processed trying to decompose them into the smallest chunks. • Communication: once identified potential parallel tasks, communication needs between them must be analyzed. • Binding: given that the cost of communications is high in terms of global execution time, the formerly identified tasks have to merge partially in order to balance computation and communication. • Mapping: once the program’s structure is settled, the recently generated processes have to be spread across the computers available. The strategy to be adopted differs according to Introduction LABORATORY GUIDE 6 the fragmentation way. As a rule of thumb, there should be at least as many processes as computers are available in order to prevent anyone being unused. If all computers are equal, it would be recommendable to make as create as many processes as computers. If not, the most powerful computers can host a higher number of processes. It is also possible to assign processes to nodes on the go, thus balancing processors’ load dynamically. Depending on several aspects, being the type of computer one of the most relevant, parallel programming admits different approaches: • Message passing: especially indicated for distributed memory computers, can be used on any hardware platform. • Shared memory: suitable only for shared memory environments. • Hybrid programming: a combination of the two previous. It is meant to optimize performance when both shared and distributed memory schemes are present. This scenario is very common in recent days. Modern clusters and MPPs are integrated by multicore memory sharing nodes. In this course we will assume that the student is familiar enough with message passing programming. More precisely, the concepts given in the Bachelor Degree on Computer Science about MPI programming are considered as known. Otherwise it is highly recommended for the student to go through the MPI Programming Fundamentals course. At least from activity 0 to activity 5. LABORATORY GUIDE Activity 1: MPI Matrix Multiplication 7 Activity 1: MPI Matrix Multiplication OBJETIVES  Apply previously acquired knowledge to develop a bit more complex program intended to be used as a benchmark to measure system performance. THEORETICAL CONCEPTS No new concepts will be introduced in this chapter since it is meant to exploit those already learned. As obvious, not all aspects of MPI development environment have been exposed and nor our application program is expected to find the most optimal solution but quite a good job is possible though. However, it may be helpful to introduce some additional information about the functions we already know. Function MPI_Recv returns a MPI_Status type parameter that we haven’t used so far. It is a structure integrated by 3 elements: MPI_SOURCE, MPI_TAG & MPI_ERROR. The first one contains the Rank of the sender process. If the message was received under MPI_ANY_SOURCE it can be necessary to find out who sent it later on in the program. The second one returns the message’s tag. If it was received under MPI_ANY_TAG, it could be interesting to get to know the tag’s value as well. The third one returns an error code. We won’t deal with error codes in this exercise. PRACTICAL EXERCISE We will program a parallel matrix multiply. It is the student’s decision how to scatter calculations among all the processes. The size of the matrices (square) must be configurable. Dynamic memory allocation is strongly recommended so no limits to the size of the matrices are imposed. Process 0 will initialize the operand matrices with any value (random, loop, etc). Data type will be float. In a first stage, multiplication results will be displayed to check correctness. Once the program has been validated, result printing must be removed to allow matrix size to grow. Execution time has to be displayed in all cases. REMARK: To combine double indexing with dynamic memory allocation for matrices, we must use double pointers. Each pointer within an array will give access to a row in a matrix: // Declare a double poiter for the matrix // This will let us refer to the elements in a [row][column] manner float **Matrix; // Initialize the double poiter to store poiters to each and every row in the matrix. Matrix = (float **) malloc(ROWS*sizeof(float *)); // We initialize each poiter to the starting poit of each row for (i=0; i< ROWS; i++) { MatriX[i] = (float *) malloc(COLUMNS*sizeof(float)); } // Now we can us [row][column] format for our matrix: Activity 1: MPI Matrix Multiplication LABORATORY GUIDE 8 for (int i=0; i<ROWS; i++) { for (int j=0; j<COLUMNS; j++) { Matrix[i][j] = 0.0; } } However, this dynamic allocation procedure does not guarantee that rows in the matrix are contiguous in memory. This can be necessary for sending functions in our program. We should send data row by row in that scenario. If we want to keep double indexing while adding contiguity, we will have to proceed as follows: // Declare a double poiter for the matrix // This will let us refer to the elements in a [row][column] manner float **Matrix; // Initialize the double poiter to store poiters to each and every row in the matrix. MatriX = (float **) malloc(ROWS*sizeof(float *)); // Declare a new pointer to allocate memory space for the whole matrix. float *Mf; // Initialize the pointer that will guarantee consecutive location of all rows Mf = (float *) malloc(ROWS*COLUMNS*sizeof(float)); // We initialize each poiter to the starting poit of each row. for (i=0; i< ROWS; i++) { MatriX[i] = Mf + i* COLUMNS; } // Now we can us [row][column] format for our matrix: for (int i=0; i<ROWS; i++) { for (int j=0; j<COLUMNS; j++) { Matrix[i][j] = 0.0; } } It is now important to notice that this alternative leads to the use of Matrix[0] as the starting address of the data stored in the matrix. QUESTIONS • In order to multiply A×B matrix A can be delivered to all processes whilst matrix B se can be distributed in columns. Think of a different option. • Would it be possible to avail of the power of Cartesian topology to facilitate the resolution of this exercise? • The need to broadcast one of the matrices slows program execution. Think of a different solution to avoid delivering so much information. Try to guess what the performance of this new option would be compared with the current program. LABORATORY GUIDE Activity 2: Performance assessment 9 Activity 2: Performance Assessment OBJECTIVES  To measure system’s performance in various circumstances.  To learn how to estimate system’s power and how to exploit it. A compromise between learning effort and code optimization must be obtained. THEORETICAL CONCEPTS In this chapter some common performance related concepts are presented: • Degree of parallelism (DOP): Number of processors used to run a program in a precise moment on time. The curve, DOP = P(t), representing the degree of parallelism as a function of time is called parallelism profile of the program. It doesn’t need to match the number of processors available (n). For the following definitions we will assume that there are more processors than necessary to reach the maximum degree of parallelism admitted by a program: máx{P(t)} = m < n. • Total amount of work: Being ∆ the computation capacity of a single processor, given either in MIPS or MFLOPS, and assuming all processors to be equal, it is possible to measure the amount of work carried out between time instant tA and tB from the area under the parallelism profile as: ∫⋅⋅ ∆ = B A t t dtPW )( . Usually the parallelism profile is a discrete graph (figure 3), so the total amount of work can be computed as: ∑ = ⋅⋅∆= m ii tiW 1 . Where ti is the time span when the degree of parallelism is i, being m the maximum degree of parallelism all over the program’s execution time. According to this, the sum of the different time intervals is equal to the program’s execution time: AB m iitt t−= ∑ =1 . • Average parallelism: Is the arithmetic mean of the degree of parallelism along time: Activity 3: Introduction to Hybrid Programming LABORATORY GUIDE 16 for(i=0;i<n;i++){ Operations to be performed } } The “n” operations to be performed will be scattered among the N threads. That will hopefully result in a reduction of execution time in case of multicore/multithreaded processors. This is a shared memory environment but, where are the shared variables? Variables declared outside the parallel region are shared. Variables declared inside the parallel region are private to each thread. Still it is possible to turn a shared variable into a private one: #pragma omp parallel num_threads (N) private (j) { #pragma omp for for(i=0;i<n;i++){ Operations to be performed on variable j } } In this case, each thread will have its own copy of “j” even though it was declared outside the region but, what would be j’s value on each thread? In the previous piece of code “j” is not initialized regardless the value it might have before the region. If we want use its previous value to initialize each thread’s copy: #pragma omp parallel num_threads (N) firstprivate (j) { #pragma omp for for(i=0;i<n;i++){ Operations to be performed on variable j } } Likewise, we may need the master thread to be aware of the changes suffered by “j” inside the region once it finishes. We can force the value of “j” to be the last one taken inside the region: #pragma omp parallel num_threads (N) firstprivate (j) lastprivate (j) { #pragma omp for for(i=0;i<n;i++){ Operations to be performed on variable j } } To end up this brief introduction, we will have a look at an additional capability of OpenMP. It won’t be hard to understand since there is an equivalent one in MPI we have already used. This is the reduction operation. It applies to a situation where a shared variable is being modified into different values by different threads. Sometimes the final value of this variable LABORATORY GUIDE Activity 3: Introduction to Hybrid Programming 17 has to be obtained from a combination of the values generated by the different threads. Let’s have a look at the example: #pragma omp parallel num_threads (N) { #pragma omp for reduction(+:sum) for (i=0;i<n;i++){ sum=sum+(a[i]); } } It is obvious that we intend to obtain a final value of “sum” which should be the result of the “n” sums performed on it. The reduction clause will take the last value generated by each thread and then perform a final sum on all of them. To make this possible, a private copy of the shared variable is generated on each thread. PRACTICAL EXERCISE Take again the matrix multiply program and conduct the following experiments: 1. Perform the multiplication on 5000x5000 matrices. Launch two processes. 2. Do the same with as many processes as processor cores available. 3. Do it again with one more process than cores. 4. Now adapt your program to the hybrid programing paradigm launching two processes and splitting the working one in as many threads as cores minus one. 5. Run the same hybrid program with as many threads as cores. Compare the time spent by the different experiments and answer the following QUESTIONS • Which programming paradigm provides de highest performance? • Is it more optimal to run only one process/thread on each core or it turns out that process 0 must share core with another process/thread? • Are these results what we could expect? Why? Activity 4: Hybrid Programming LABORATORY GUIDE 18 Activity 4: Hybrid Programming OBJECTIVES  Understand some work scheduling options in order to optimize execution time. THEORETICAL CONCEPTS Synchronization. The default synchronization procedure introduces a barrier at the end of the parallel region so execution does not continue until all threads reach that point. This is a sensible thing to do but, in certain cases, it may be useful to avoid that constraint. This can be done by means of the “nowait” clause. #pragma omp parallel num_threads (N) { #pragma omp for nowait for(i=0;i<n;i++){ Operations to be performed on variable j } } In this particular example it doesn’t make any difference but, in case we had another parallel loop right after, it would save time if some threads could enter it as soon as possible. Scheduling. So far we have assumed that the amount of work to be done is delivered to the different threads in a fair manner. That’s right but, even in this case there could be different possibilities that result in significant performance variations. The default scheduling policy divides the number of iterations by the number of threads thus giving each thread the same amount of work if possible. This work is assigned prior execution and no changes are made at run time. It is possible to specify different work “chunks”. In this case each particular implementation decides how to allocate chunks on threads. #pragma omp parallel num_threads (N) { #pragma omp for schedule(static,10) for(i=0;i<n;i++){ Operations to be performed on variable j } } LABORATORY GUIDE Activity 4: Hybrid Programming 19 In this example chunks of 10 iterations are delivered. The last chunks are made smaller when necessary. Static policies do not allow to dynamically assigning pieces of work to threads as they finish their previously assigned one. This results in a loss of efficiency that should be avoided. Dynamic policies can be applied to do so. #pragma omp parallel num_threads (N) { #pragma omp for schedule(dynamic,10) for(i=0;i<n;i++){ Operations to be performed on variable j } } In this example, threads get new chunks as soon as they finish their current calculation. PRACTICAL EXERCISE We will continue the experiments done on the previous exercise. We already have the results obtained from the default static scheduling. Now we will add these new ones: • Try again the static scheduling but specifying a chunk size of 10. • Then try chunk size 100. • Now shift to dynamic scheduling with chunk size 10. • Try again dynamic with chunk size 100. Compare all results to see which the best policy is and try to explain why. Work with the number of threads that proved to be the best option in the previous exercise. REFERENCES: OPENMP APPLICATION PROGRAM INTERFACE. Available at: http://www.openmp.org/mp-documents/spec30.pdf Activity 5: MPI vs OpenMP LABORATORY GUIDE 20 Activity 5: MPI vs OpenMP OBJECTIVES  In hybrid programming many different number of threads and processes may be launched. We will try to find out which is the best combination.  Message passing and shared memory involve different programming techniques and a distinct use of hardware resources. We need to know which one is more efficient and then more convenient. THEORETICAL CONCEPTS No additional theoretical discussion will be introduced for this exercise. PRACTICAL EXERCISE We will launch a battery of test meant to fulfill the first of the objectives already stated: • Repeat the matrix multiplication on two 5000x5000 matrices with two MPI processes in the local machine. • Launch as many MPI processes as cores are available. • Launch as many MPI processes as cores are available plus one. • Back to two MPI processes split the working one (rank 1) into as many threads as cores available minus one. • Split rank 1 into as many threads as cores are available so one of its threads will share a core with rank 0 process. Compare all results to see which the best policy is and try to explain why. Now we will address the second objective. Use the 5000 x 5000 case again: • Launch as many processes as processors available plus one and split the working processes into as many threads as cores available. • Launch as many processes as cores available plus one (no shared memory this time). Compare the results and try to explain them. See references to find answers. REFERENCES: Comparing the OpenMP, MPI, and Hybrid Programming Paradigms on an SMP Cluster Gabriele Jost and Haoqiang Jin and Dieter An Mey and Ferhat F. Hatay NAS Technical Report NAS-03-019, November 2003. LABORATORY GUIDE Activity 6: Submitting Jobs to the cluster. 21 Activity 6: Submitting jobs to a cluster OBJETIVES  Get to know how Jobs are submitted to a computation cluster.  Understand the differences between a local working environment and a cluster architecture THEORETICAL CONCEPTS The Jobs we are about to submit to the cluster are no different from those we have been working with so far. They will be MPI programs mainly derived from the matrix multiply application we are using as a benchmark. We will work in Windows 8.1 using the user roles previously generated within the ARAVAN workgroup and also within the HPC (High Performance Computing) cluster. The user will be allowed to launch jobs to the cluster. From now we are going to use the Microsoft MPI implementation: MS-MPI. The tool used to submit these Jobs is the “Job Manager” and it is part of the client tools installed by the HPC PACK 2012 R2. Before we can send jobs to execution there are a few issues we have to deal with: 1. We won’t have a GUI. Initialization information will be parsed to the applications from the command line. Other information needed at run time has to be provided within a file. Therefore it will be necessary to adapt our programs to these situations in certain cases. Concerning the matrix multiply program we have developed, matrix size will be introduced as an initialization parameter from command line. A code line simliar to: “size = atoi (argv[1]);” will provide the numerical value of this parameter so it can be used within the program. 2. Program’s output will be redirected to a text file we will have to open once the program has finalized to see the results. 3. The job manager will consider the program’s execution unsuccessful unless it returns a zero code. We can write “exit (0)” at the end of the program to do so. It is defined within <stdlib.h>. Local job generation with Job Manger. A job is integrated by a number of tasks. Tasks are user applications meant to be executed by the system. We mean to launch jobs comprising one single task: our MPI application. In this case we can use the option “Single Task Job” to make the process simple. Activity 6: Submitting Jobs to the cluster. LABORATORY GUIDE 22 Figure 6.1. Single task job configuration. We have to select the working directory. In this case we introduce the folder where the input and output text files are to be placed. We also introduce the names for these files. If no input data is required the “Standard input” field may be left blank. On the command line we describe the task to be performed, a parallel MPI application in this case: “mpiexec –n 4 c:\mpiapps\MPIapp1.exe”. It doesn’t need to be located in the working directory. The “-n 4” parameter tells the system to launch 4 processes. Parametric sweep jobs. In many real situations, tasks are not performed individually but rather in a combined manner so results can be analyzed and compared. As a matter of fact, we usually launch many executions of our matrix multiply program to see how different configurations and sizes affect execution time. It is possible to launch a job for each case but it would be more efficient to launch them all together. This is what the “Parametric sweep job” option makes possible. LABORATORY GUIDE Activity 6: Submitting Jobs to the cluster. 23 Figure 6.2. Parametric sweep job configuration. In this example we have set the parameter to vary from 1 to 5 incrementing one by one. As a result, 5 tasks will be conducted, one for each of its values. The asterisk used to place the parameter in the command line is also placed within the names of the text files so each task is linked to its own output file. In this example we have used the parameter to modify the command line argument parsed to the program but it can connected with any other aspect of the information provided in the command line. For instance, we could vary the number of processes to be launched instead: “mpiexec –n * c:\ruta\multimatriz 5000”. We could provide more than one asterisk in the same command line but it is very unlikely that the same values make sense in different positions. Netting parameters within the same task is not permitted. Job generation with Job Manger for the cluster. Generating jobs for the local node or for the cluster is conceptually the same, since the former is just a section of the later. Nevertheless is important to remark in this section some settings to be made: • Folder and subfolder sharing. • Working directory configuration. • Node selection. Activity 6: Submitting Jobs to the cluster. LABORATORY GUIDE 24 For the applications to be executed by remote nodes, the working directory must be shared. We can use Windows Explorer to edit the properties of the folder containing the working directory and then share it. Figure 6.3. Sharing the working directory. Users meant to execute the application must have the appropriate rights. If the job is to be executed by other nodes, its path must be known under a common format. UNC (https://msdn.microsoft.com/en-us/library/gg465305.aspx) is the one accepted for this purpose. It is use to declare the path for the working directory. The rest of paths: input and output files and the application itself are referred to the working directory as a relative path. Figure 6.4 shows how to make these settings. LABORATORY GUIDE Activity 6: Submitting Jobs to the cluster. 25 Figure 6.4. Configuratiopn of the shared working directory. In this particular case the application’s whole path would be: TE-C-24\c:\cluster\programas\Commandexample7\x64\Release\Commandexample764.exe, where “150” is a command line argument for the application. When configuring a new job, the “Resource Selection” option will display the available nodes on the cluster so we can select the desired ones. Activity 9: Job scheduling III. LABORATORY GUIDE 32 Activity 9: Job scheduling III OBJETIVES  Understanding preemption.  Checking the influence of preemption on system performance. THEORETICAL CONCEPTS Preemption allows higher priority jobs to interrupt lower priority ones. As shown before, this can be done in different ways. Since our goal remains system performance, higher priority should be given so the overall execution time is minimized. PRACTICAL EXERCISE Group the tasks launched in previous scheduling activities in two jobs. On one job the tasks comprising less processes will be placed and this job will be given the highest priority. The other job, with the lowest priority will entail the rest of the tasks. Under both queued and balanced scheduling policies, repeat the usual experiments trying the different preemption options available. Build up again the tables and compare results. Decide what preemption policy is the most advisable for this type of workload. Compare the results obtained under queued scheduling policy with and without the clicks on the “Adjust resources automatically” options. Compare the results obtained under balanced scheduling policy using the different biasing options available. LABORATORY GUIDE Activity 10: Performance competition. 33 Activity 10: Performance competition. OBJETIVES  Making the best scheduling decisions. THEORETICAL CONCEPTS No theoretical concepts are introduced in this activity. PRACTICAL EXERCISE For a given matrix multiplication application (the same for all participants), each one will make what are expected to be the best scheduling decisions. This will include using jobs, tasks or both. Once they are made, the usual experiments will be conducted and the overall execution times compared in order to find out what were actually the best scheduling options. In your report include your decisions, your results and compare them with the best performer. Explain why you think your decisions were not the best. If you are the best performer, congratulations, you will save some work. Appendix A: Installing DeinoMPI LABORATORY GUIDE 34 Appendix A: Installing DeinoMPI DeinoMPI in an implementation of the standard MPI-2 for Microsoft Windows derived from Argonne Nacional Laboratory’s MPICH2. System requirements: • Windows 2000/XP/Server 2003/Windows 7 • .NET Framework 2.0 Installation DeinoMPI has to be downloaded and then installed in all computers in the cluster. The installation process is the same in all nodes. It requires administrator privileges for installation but all users can execute it afterwards. Once it is installed folder \bin has to be added to the path. Note: make sure Deino’s version matches the operating systems requirements (32 or 64 bits). Configuration Once the software has been installed, each user will need to create a “Credential Store”. It is used to launch routines in a secure manner. Mpiexec will not execute any of them without this “Credential Store”. The graphic environment will show the user this option in the first execution. Launching Jobs Once again, both the graphic environment and the command line are valid. Graphic Environment This tool can be used to launch MPI processes, manage the “Credential Store”, search for computers within the local network that have MPI installed, verify mpiexec entries to diagnose common problems, and go to the DeinoMPI web site to look for help and documentation. Mpiexec tab It is the main page and is used to launch and manage MPI processes. LABORATORY GUIDE Appendix A: Installing DeinoMPI 35 Figure A1. Mpiexec tab. These are the main elements of this tab: • Application: o The MPI application’s path is introduced here. The same path will be taken by default in all nodes within the cluster so it is recommendable to copy the .exe file in the same folder in all of them. o If a network folder is specified, it is necessary to have sufficient privileges in the server. o The “application” button can be used to locate the .exe file. • Execute: the program selected in the application dialog is launched when this button is pressed.. • Break: aborts program execution. • Number of processes: Sets the number of processes to be launched. • Credential Store Account: Sets the active user of the Credential Store. • Check box “more options”: It expands/contracts the options area. • Hosts: Introduce here the list of hosts where you want the processes to run. Host names are separated by blanks. To execute the program in the local machine only, keep the default option “localonly” active or write down its name on this list Appendix A: Installing DeinoMPI LABORATORY GUIDE 36 Credential Store Tab. This tab is used to manage user’s credential store. If no credential store has been created so far, select “enable create store options” check box to make remaining options available. They are hidden by default since they are only used the first time Deino is initiated. Figure A2. Credential Store tab including all options. In order to create a credential store, the “enable create store options” check box must be selected. Three possibilities arise: • “Password”: o If this option is selected, the credential store will be protected from access by a password. It is the most secure option but forces the user to introduce the password any time a job has to be launched. o If “No password” is selected, the use of MPI is easier but more vulnerable. Without a password any program launched by the user can access the credential store which is not really a problem provided no malicious software is being used. o Even with this “No password” option active, the credential store is not available to other users if the encryption option is selected. • “Encryption”: LABORATORY GUIDE Appendix A: Installing DeinoMPI 37 o “Windows ProtectData API” allows encryption of the credential store using the encryption scheme used by Windows for the current user. This ensures the credential store will only be available when the user is validated. o If a password is selected the “symmetric key” encryption format can be chosen. This encryption is not specific to the user so other user knowing the password could access the store. o The “no encryption” option is not recommended since it stores the credential store in a plain text file accessible to all users. • “Location”: o Take the “Removable media” option to save the store in an external device such as a memory stick. In this case, jobs can only be launched when the device is attached to the computer. This can be the safest option since the user can decide when the credential store is present. Combined with the use of a password and its encryption it can be protected even against loss or robbery. o The “Registry” option moves the “Credential Store” to the Windows registry. o Finally, it can be stored in the “Hard drive” which turns out to be the most common decision. Cluster tab In this tab, the computers in the cluster are displayed and the DeinoMPI version installed in each of them. Figure A3. Cluster tab – Big icons view. Appendix A: Installing DeinoMPI LABORATORY GUIDE 38 More hosts can be added writing down their name of can be found automatically within the selected domain. Deino MPI manual. Available at: http://mpi.deino.net/manual.htm LABORATORY GUIDE Appendix B: Project Configuration in Visual Studio 2010 39 Appendix B: Project Configuration in Visual Studio 2010 In this section we will describe the same configuration process but for the 2010 version of Microsoft Visual Studio. Configuration in more recent versions of Visual Studio is analogous. • Generate a new project and solution. They may have both the same name: • Set it as empty project: Appendix B: Project Configuration in Visual Studio 2010 LABORATORY GUIDE 40 • Once created both the Project and solution, add a code file as new item: LABORATORY GUIDE Appendix B: Project Configuration in Visual Studio 2010 41 • Now, and never before, the Project settings are entered (“Properties”): 1. In the C/C++ section we must enter the route to the folder where the header MPI files are located (“Additional Include Directories”). By default the \Archivos de Programa (x86)\DeinoMPI\include is assumed: 2. In the Linker section we must enter the route to the folder where the MPI libraries are located (“Additional Library Directories”). By default \Archivos de Programa (x86)\DeinoMPI\lib is assumed: Appendix C: Configuration of MS - MPI LABORATORY GUIDE 48 3. Set also the new library file. 4. When all these parts have been configured the solution can be built as usual. In order to execute the program, the .exe file and MPI’s launcher must be in the same folder or either the path configured accordingly. The launcher is mpiexec.exe and is placed in Program Files > Microsoft MPI > bin. Write down mpiexec –n np program.exe, where np is the number of processes to be launched. LABORATORY GUIDE Appendix B: Project Configuration in Visual Studio 2010 49