Program

We introduce the concept of parallel patterns as a viable medium to improve both parallel programmers' productivity and efficiency of parallel applications. We briefly discuss possibilities offered by patterns relative to program refactoring, autonomic management of non-functional properties as well as heterogeneous hardware targeting. We will take into account different kinds of heterogeneity, ranging from accelerator usages to exploitation of classical HPC architectures, small size parallel architectures, as well as cloud and edge. Pointers to notable examples of parallel programming environments somehow providing or building on parallel patterns will be also discussed.

The proliferation of diverse computing devices, from low-power IoT sensors to massive cloud clusters, has yielded a continuous spectrum of computation - the compute continuum. Structured programming models and paradigms, like MapReduce, are pivotal in enabling large-scale distributed systems, but they do not always compose well across this continuum. As computing expands into a continuous spectrum from IoT to the cloud, the availability of decentralized programming models could be beneficial. Early paradigms scale clusters but not the wider, heterogeneous, continuum. Actor models, with message-passing and encapsulated states, along with reactive systems could be a remedy for this. They enable adaptability, openness, modular philosophies, and resilience across diverse devices and distributions. Current tools now bring these benefits but often require programmers to change their approach to application development. This lesson builds an understanding of the progression that yielded today's actor-based approaches, reactive systems, and the path forward. It aims to spark debate around how our field must advance tools and methodologies to succeed in an age of spreading data decentralisation and distributed computation.

The proliferation of diverse computing devices, from low-power IoT sensors to massive cloud clusters, has yielded a continuous spectrum of computation - the compute continuum. Structured programming models and paradigms, like MapReduce, are pivotal in enabling large-scale distributed systems, but they do not always compose well across this continuum. As computing expands into a continuous spectrum from IoT to the cloud, the availability of decentralized programming models could be beneficial. Early paradigms scale clusters but not the wider, heterogeneous, continuum. Actor models, with message-passing and encapsulated states, along with reactive systems could be a remedy for this. They enable adaptability, openness, modular philosophies, and resilience across diverse devices and distributions. Current tools now bring these benefits but often require programmers to change their approach to application development. This lesson builds an understanding of the progression that yielded today's actor-based approaches, reactive systems, and the path forward. It aims to spark debate around how our field must advance tools and methodologies to succeed in an age of spreading data decentralisation and distributed computation.

There is an ever-increasing need for more application performance and energy efficiency. However, since 2005 computing platforms have been providing more performance only by increasing the number of processors (not anymore by just increasing clock frequency). We currently range from multi-core encompassing even 128 cores to GP-GPU with more than 10000 cores on the single chip and to the aggregation of these resources connected via high-speed interconnects. Parallel programming permits us to express an algorithm to exploit what the technology is providing us properly. But: what are the principles of parallel programming that best allow (or limit) maximum performance? This course covers several foundational topics central to parallel computing, starting from the basic methods to identify performance bottlenecks properly and reviews the practical means to program a parallel machine by using the most used standards such as OpenMPI, OpenMP, CUDA We analyze these concepts through several hands-on examples.

Introductory course
The GPU Programming course is designed to provide participants with a practical understanding of GPU programming and its applications. The course will cover essential topics including CUDA programming, parallel algorithms, optimization strategies, and performance analysis. The program will be split into 6 hours, with a hands-on session included. The schedule is as follows: Morning Session: Introduction to GPU computing and CUDA programming. Participants will learn the basics of GPU architecture and programming, including CUDA memory model and thread hierarchy. Afternoon Session: Hands-on session in which participants will work on practical exercises to solidify their understanding of the concepts covered in the morning session. The session will include debugging and optimization techniques to improve the performance of GPU kernels. By the end of the course, participants will have a good understanding of GPU programming concepts, and will have the ability to write and optimize GPU kernels for their own applications."
Advanced
"The Advanced GPU Programming session is designed for participants who already have a background in GPU programming and CUDA. The course will cover advanced topics such as optimization techniques, performance analysis, and parallel algorithms for specific application domains. The technical program is as follows Morning Session: Advanced optimization techniques for GPU kernels. Participants will learn how to use profiling tools and optimize the memory access patterns and the use of shared memory to improve the performance of their GPU kernels. Afternoon Session: Hands-on session in which participants will work on case studies and real-world examples. The session will include the implementation of parallel algorithms for specific application domains such as machine learning, and scientific computing or data analytics. By the end of the course, participants will have a deep understanding of advanced GPU programming concepts and will have the ability to optimize and implement parallel algorithms on GPU for their specific application domains."

The evolution of High-Performance Computing (HPC) must face several obstacles, including power/thermal management of the cores and the presence of heterogeneous computing platforms. Many of the problems are common to the entire computing continuum but, despite the presence of similar criticalities, the solutions adopted can have very specific peculiarities. For instance, at the bottom of the computing stack, energy aware management is of paramount importance, while moving toward the server side, power and thermal management plays the main role. In this class, the fundamentals on the sources and models of power consumption will be discussed, with the goal to better understand the main strategy to orchestrate the applications and to keep under control the power consumption and the temperature of the platforms during the computation. Specific deep dives on best practices coming from the state of the art generated during EU-funded projects will be provided, to stimulate the discussion among the students and to create links for further investogations.

Students will be welcomed "inside" an HPC cluster. A short seminary will show everything there is to know about the environment and the practical usage of an important supercomputer like those hosted by CINECA: from login to filesystem and storage areas, from submitting batch jobs to interactive access in compute nodes, from serial and parallel compiling to take advantage of the accelerators. If time allows for it, the students will be "users" for a day and try small exercises inside one of Cineca's clusters, so it will be possible for them to put in practice the content of the lesson.

During this presentation students will receive a short summary of what is CINECA, our current and future HPC cluster that will be hosted in our infrastructures, the ICSC centre and how to obtain computational resources on them. Special interest will be devoted to the Leonardo Supercomputer and the possible usage of it.