Hardware Design for Machine Learning Acceleration, 6 credits
Hårdvarudesign för acceleration av maskininlärning, 6 hp
TSEA85
Main field of study
Electrical EngineeringCourse level
Second cycleCourse type
Programme courseExaminer
Jose Nunez-YanezDirector of studies or equivalent
Anders NilssonEducation components
Preliminary scheduled hours: 42 hRecommended self-study hours: 118 h
Main field of study
Electrical EngineeringCourse level
Second cycleAdvancement level
A1FCourse offered for
- Master's Programme in Electronics Engineering
- Master of Science in Applied Physics and Electrical Engineering
- Master of Science in Applied Physics and Electrical Engineering - International
- Master of Science in Computer Science and Engineering
Prerequisites
Switching Theory and Logical Design, Computer Hardware and Architecture, Digital Project Laboratory, C/C++ programming
Intended learning outcomes
The course intends to give hands-on experience on the design of an advanced machine learning accelerator using reconfigurable hardware with a focus on performance, debugging, , memory and interaction between the hardware and software.
After completion of course the student shall be able to:
- Design and integrate hardware accelerators for machine learning applications.
- Use high-level languages C/C++ for system-on-chip design.
- Integrate a system-on-chip of IP blocks to solve a specific problem.
- Debug and estimate the performance of the accelerator via simulation.
- Implement the design in an FPGA board, test functionality using testbenches, measure performance and analyze performance bottlenecks.
Course content
In this course we will explore the architecture and features of modern FPGAs (Field Programable Gate Arrays) together with their application to machine learning and artificial intelligence hardware acceleration. FPGAs are a form of reconfigurable hardware in which the underlaying logic adapts to fit the algorithmic problem and they can achieve higher performance and better energy efficiency than alternative CPU/GPU hardware in certain applications. We will focus on the architecture of system-on-chip devices that combine a processor and FPGA fabric in the same chip such as the Xilinx/AMD Zynq and Versal families. While the Zynq family is characterized by the presence of programmable logic and processing system in the same device, the Versal family adds so called intelligent engines as a 2-dimensional array of VLIW (Very Long Instruction Word) programmable processors. This makes these devices a highly scalable and heterogeneous platform that requires significant engineering software and hardware skills. We will learn how to use C-based high-level synthesis to replace traditional Verilog/VHDL RTL design to create fast and efficient hardware accelerators for neural network operators within the machine learning and artificial intelligence fields. The course will cover different hardware optimization strategies and how C-based pragmas are used to control the output generated by high-level synthesis. In the labs the VITIS HLS and VIVADO synthesis and implementation tools will target a SoC Zynq FPGA that also includes an ARM-based processing system. The ARM processor acts as a host and sends and collects results from the hardware accelerator. The course requires previous knowledge on computer architecture, digital design and C/C++ based programming. Some previous experience with Python would be useful. C/C++ is used as the design language for the accelerator using state-of-the-art high-level-synthesis compilers. The accelerator is then exported as an IP block in VHDL/Verilog and integrated with other components including the ARM processor to form a functional system-on-chip. Initial simulations followed by FPGA implementation are used to verify the correct functionality and the obtained acceleration level.
Teaching and working methods
Lectures, quiz, project and laboratory work. The quiz is organized during the teaching time to verify that initial lab exercises and lectures have been completed and understood. During the project work hardware acceleration functions are designed and tested. A final report documents the system characteristics, performance and functionality.
Examination
LAB1 | Laboratory Work | 2 credits | U, G |
PRA1 | Project | 4 credits | U, G |
Grades are given as ’Fail’ or ’Pass’. Both modules need a ‘Pass’ for the overall grade to be ‘Pass’
Grades for examination modules are decided in accordance with the assessment criteria presented at the start of the course.
Grades
Two-grade scale, U, GOther information
About teaching and examination language
The teaching language is presented in the Overview tab for each course. The examination language relates to the teaching language as follows:
- If teaching language is “Swedish”, the course as a whole could be given in Swedish, or partly in English. Examination language is Swedish, but parts of the examination can be in English.
- If teaching language is “English”, the course as a whole is taught in English. Examination language is English.
- If teaching language is “Swedish/English”, the course as a whole will be taught in English if students without prior knowledge of the Swedish language participate. Examination language is Swedish or English depending on teaching language.
Other
The course is conducted in such a way that there are equal opportunities with regard to sex, transgender identity or expression, ethnicity, religion or other belief, disability, sexual orientation and age.
The planning and implementation of a course should correspond to the course syllabus. The course evaluation should therefore be conducted with the course syllabus as a starting point.
The course is campus-based at the location specified for the course, unless otherwise stated under “Teaching and working methods”. Please note, in a campus-based course occasional remote sessions could be included.
Department
Institutionen för systemteknikCourse literature
Additional literature
Books
- Crockett H Louise (Author), Northcote David (Author), Ramsay Craig (Author), (2019) Exploring Zynq MPSoC: With PYNQ and Machine Learning Applications 1st Strathclyde Academic Media
ISBN: 978-0992978761
This book introduces the Zynq MPSoC (Multi-Processor System-on-Chip), an embedded device from Xilinx and the PYNQ platform to design machine learning applications.
https://www.zynq-mpsoc-book.com/
Websites
- Intro to FPGA design with HLS https://docs.amd.com/v/u/en-US/ug998-vivado-intro-fpga-design-hls
- PYNQ (Python on Zynq) http://www.pynq.io/
- VITIS HLS synthesis guide https://docs.xilinx.com/r/en-US/ug1399-vitis-hls
- Zynq SoC https://www.xilinx.com/products/silicon-devices/soc/zynq-7000.html#documentation
Code | Name | Scope | Grading scale |
---|---|---|---|
LAB1 | Laboratory Work | 2 credits | U, G |
PRA1 | Project | 4 credits | U, G |
Grades are given as ’Fail’ or ’Pass’. Both modules need a ‘Pass’ for the overall grade to be ‘Pass’
Grades for examination modules are decided in accordance with the assessment criteria presented at the start of the course.
Course syllabus
A syllabus must be established for each course. The syllabus specifies the aim and contents of the course, and the prior knowledge that a student must have in order to be able to benefit from the course.
Timetabling
Program courses are timetabled after a decision has been made for this course concerning its assignment to a timetable module. Single subject courses can be timetabled at other times.
Interruption in and deregistration from a course
The LiU decision, Guidelines concerning confirmation of participation in education, Dnr LiU-2020-02256 (https://styrdokument.liu.se/Regelsamling/VisaBeslut/764582), states that interruptions in study are to be recorded in Ladok. Thus, all students who do not participate in a course for which they have registered are therefore obliged to report the interruption so that this can be noted in Ladok. Deregistration from or interrupting a course is carried out using a Web-based form.
Cancelled courses and changes to the course syllabus
Courses with few participants (fewer than 10) may be cancelled or organised in a manner that differs from that stated in the course syllabus. The Dean is to deliberate and decide whether a course is to be cancelled or changed from the course syllabus. For single subject courses, the cancellation must be done before students are admitted to the course (in accordance with LiUs regulation Dnr LiU-2022-01200, https://styrdokument.liu.se/Regelsamling/VisaBeslut/622645).
Guidelines relating to examinations and examiners
For details, see Guidelines for education and examination for first-cycle and second-cycle education at Linköping University, Dnr LiU-2023-00379 (http://styrdokument.liu.se/Regelsamling/VisaBeslut/917592).
An examiner must be employed as a teacher at LiU according to the LiU Regulations for Appointments, Dnr LiU-2022-04445 (https://styrdokument.liu.se/Regelsamling/VisaBeslut/622784). For courses in second-cycle, the following teachers can be appointed as examiner: Professor (including Adjunct and Visiting Professor), Associate Professor (including Adjunct), Senior Lecturer (including Adjunct and Visiting Senior Lecturer), Research Fellow, or Postdoc. For courses in first-cycle, Assistant Lecturer (including Adjunct and Visiting Assistant Lecturer) can also be appointed as examiner in addition to those listed for second-cycle courses. In exceptional cases, a Part-time Lecturer can also be appointed as an examiner at both first- and second cycle, see Delegation of authority for the Board of Faculty of Science and Engineering.
Forms of examination
Principles for examination
Written and oral examinations and digital and computer-based examinations are held at least three times a year: once immediately after the end of the course, once in August, and once (usually) in one of the re-examination periods. Examinations held at other times are to follow a decision of the faculty programme board.
Principles for examination scheduling for courses that follow the study periods:
- courses given in VT1 are examined for the first time in March, with re-examination in June and August
- courses given in VT2 are examined for the first time in May, with re-examination in August and January
- courses given in HT1 are examined for the first time in October, with re-examination in January and August
- courses given in HT2 are examined for the first time in January, with re-examination in March and in August.
The examination schedule is based on the structure of timetable modules, but there may be deviations from this, mainly in the case of courses that are studied and examined for several programmes and in lower grades (i.e. 1 and 2).
Examinations for courses that the faculty programme board has decided are to be held in alternate years are held three times during the school year in which the course is given according to the principles stated above.
Examinations for courses that are cancelled or rescheduled such that they are not given in one or several years are held three times during the year that immediately follows the course, with examination scheduling that corresponds to the scheduling that was in force before the course was cancelled or rescheduled.
When a course, or a written or oral examination (TEN, DIT, DAT, MUN), is given for the last time, the regular examination and two re-examinations will be offered. Thereafter, examinations are phased out by offering three examinations during the following academic year at the same times as the examinations in any substitute course. If there is no substitute course, three examinations will be offered during re-examination periods during the following academic year. Other examination times are decided by the faculty programme board. In all cases above, the examination is also offered one more time during the academic year after the following, unless the faculty programme board decides otherwise. In total, 6 re-examinations are offered, of which 2 are regular re-examinations. In the examination registration system, the examinations given for the penultimate time and the last time are denoted.
If a course is given during several periods of the year (for programmes, or on different occasions for different programmes) the faculty programme board or boards determine together the scheduling and frequency of re-examination occasions.
For single subject courses, written and oral examinations can be held at other times.
Retakes of other forms of examination
Regulations concerning retakes of other forms of examination than written examinations and digital and computer-based examinations are given in the LiU guidelines for examinations and examiners, Dnr LiU-2023-00379 (http://styrdokument.liu.se/Regelsamling/VisaBeslut/917592).
Course closure
For Decision on Routines for Administration of the Discontinuation of Educational Programs, Freestanding Courses and Courses in Programs, see Dnr LiU-2021-04782 (https://styrdokument.liu.se/Regelsamling/VisaBeslut/1156410). After a decision on closure and after the end of the discontinuation period, the students are referred to a replacement course (or similar) according to information in the course syllabus or programme syllabus. If a student has passed some part/parts of a closed program course but not all, and there is an at least partially replacing course, an assessment of crediting can be made. Any crediting of course components is made by the examiner.
Registration for examination
In order to take an written, digital or computer-based examination, registration in advance is mandatory, see decision in the university’s rule book Dnr LiU-2020-04559 (https://styrdokument.liu.se/Regelsamling/VisaBeslut/622682). An unregistered student can thus not be offered a place. The registration is done at the Student Portal or in the LiU-app during the registration period. The registration period opens 30 days before the date of the examination and closes 10 days before the date of the examination. Candidates are informed of the location of the examination by email, four days in advance.
Code of conduct for students during examinations
Details are given in a decision in the university’s rule book, Dnr LiU-2020-04559 (http://styrdokument.liu.se/Regelsamling/VisaBeslut/622682).
Retakes for higher grade
Students at the Institute of Technology at LiU have the right to retake written examinations and digital and computer-based examinations in an attempt to achieve a higher grade. This is valid for all examination components with code “TEN”, “DIT” and "DAT". The same right may not be exercised for other examination components, unless otherwise specified in the course syllabus.
A retake is not possible on courses that are included in an issued degree diploma.
Grades
The grades that are preferably to be used are Fail (U), Pass (3), Pass not without distinction (4) and Pass with distinction (5).
- Grades U, 3, 4, 5 are to be awarded for courses that have written or digital examinations.
- Grades Fail (U) and Pass (G) may be awarded for courses with a large degree of practical components such as laboratory work, project work and group work.
- Grades Fail (U) and Pass (G) are to be used for degree projects and other independent work.
Examination components
The following examination components and associated module codes are used at the Faculty of Science and Engineering:
- Grades U, 3, 4, 5 are to be awarded for written examinations (TEN) and digital examinations (DIT).
- Examination components for which the grades Fail (U) and Pass (G) may be awarded are laboratory work (LAB), project work (PRA), preparatory written examination (KTR), digital preparatory written examination (DIK), oral examination (MUN), computer-based examination (DAT), home assignment (HEM), and assignment (UPG).
- Students receive grades either Fail (U) or Pass (G) for other examination components in which the examination criteria are satisfied principally through active attendance such as tutorial group (BAS) or examination item (MOM).
- Grades Fail (U) and Pass (G) are to be used for the examination components Opposition (OPPO) and Attendance at thesis presentation (AUSK) (i.e. part of the degree project).
In general, the following applies:
- Mandatory course components must be scored and given a module code.
- Examination components that are not scored, cannot be mandatory. Hence, it is voluntary to participate in these examinations, and the voluntariness must be clearly stated. Additionally, if there are any associated conditions to the examination component, these must be clearly stated as well.
- For courses with more than one examination component with grades U,3,4,5, it shall be clearly stated how the final grade is weighted.
For mandatory components, the following applies (in accordance with the LiU Guidelines for education and examination for first-cycle and second-cycle education at Linköping University, Dnr LiU-2023-00379 http://styrdokument.liu.se/Regelsamling/VisaBeslut/917592):
- If special circumstances prevail, and if it is possible with consideration of the nature of the compulsory component, the examiner may decide to replace the compulsory component with another equivalent component.
For possibilities to alternative forms of examinations, the following applies (in accordance with the LiU Guidelines for education and examination for first-cycle and second-cycle education at Linköping University, Dnr LiU-2023-00379 http://styrdokument.liu.se/Regelsamling/VisaBeslut/917592):
- If the LiU coordinator for students with disabilities has granted a student the right to an adapted examination for a written examination in an examination hall, the student has the right to it.
- If the coordinator has recommended for the student an adapted examination or alternative form of examination, the examiner may grant this if the examiner assesses that it is possible, based on consideration of the course objectives.
- An examiner may also decide that an adapted examination or alternative form of examination if the examiner assessed that special circumstances prevail, and the examiner assesses that it is possible while maintaing the objectives of the course.
Reporting of examination results
The examination results for a student are reported at the relevant department.
Plagiarism
For examinations that involve the writing of reports, in cases in which it can be assumed that the student has had access to other sources (such as during project work, writing essays, etc.), the material submitted must be prepared in accordance with principles for acceptable practice when referring to sources (references or quotations for which the source is specified) when the text, images, ideas, data, etc. of other people are used. It is also to be made clear whether the author has reused his or her own text, images, ideas, data, etc. from previous examinations, such as degree projects, project reports, etc. (this is sometimes known as “self-plagiarism”).
A failure to specify such sources may be regarded as attempted deception during examination.
Attempts to cheat
In the event of a suspected attempt by a student to cheat during an examination, or when study performance is to be assessed as specified in Chapter 10 of the Higher Education Ordinance, the examiner is to report this to the disciplinary board of the university. Possible consequences for the student are suspension from study and a formal warning. More information is available at Cheating, deception and plagiarism.
Linköping University has also produced a guide for teachers and students' use of generative AI in education (Dnr LiU-2023-02660). As a student, you are always expected to gain knowledge of what applies to each course (including the degree project). In general, clarity to where and how generative AI has been used is important.
Regulations (apply to LiU in its entirety)
The university is a government agency whose operations are regulated by legislation and ordinances, which include the Higher Education Act and the Higher Education Ordinance. In addition to legislation and ordinances, operations are subject to several policy documents. The Linköping University rule book collects currently valid decisions of a regulatory nature taken by the university board, the vice-chancellor and faculty/department boards.
LiU’s rule book for education at first-cycle and second-cycle levels is available at https://styrdokument.liu.se/Regelsamling/Innehall.
Additional literature
Books
ISBN: 978-0992978761
This book introduces the Zynq MPSoC (Multi-Processor System-on-Chip), an embedded device from Xilinx and the PYNQ platform to design machine learning applications.
Websites
Note: The course matrix might contain more information in Swedish.
I | U | A | Modules | Comment | ||
---|---|---|---|---|---|---|
1. DISCIPLINARY KNOWLEDGE AND REASONING | ||||||
1.1 Knowledge of underlying mathematics and science (G1X level) |
|
|
X
|
LAB1
PRA1
|
neural networks |
|
1.2 Fundamental engineering knowledge (G1X level) |
|
X
|
X
|
LAB1
PRA1
|
IP-block design, high level synthesis, computer architecture |
|
1.3 Further knowledge, methods, and tools in one or several subjects in engineering or natural science (G2X level) |
|
X
|
X
|
LAB1
PRA1
|
system-on-chip design |
|
1.4 Advanced knowledge, methods, and tools in one or several subjects in engineering or natural sciences (A1X level) |
|
X
|
X
|
LAB1
PRA1
|
integration of an accelerator in system-on-chip |
|
1.5 Insight into current research and development work |
|
|
X
|
LAB1
|
State-of-the-art FPGA technology and tools |
|
2. PERSONAL AND PROFESSIONAL SKILLS AND ATTRIBUTES | ||||||
2.1 Analytical reasoning and problem solving |
|
X
|
X
|
PRA1
|
Design and integration of accelerator blocks. |
|
2.2 Experimentation, investigation, and knowledge discovery |
|
X
|
X
|
PRA1
|
Performance measures and debugging using hardware and software components. |
|
2.3 System thinking |
|
|
X
|
PRA1
|
Integration of dedicated IP blocks to solve a specific problem. |
|
2.4 Attitudes, thought, and learning |
|
X
|
X
|
PRA1
|
debugging, testing, testbenches |
|
2.5 Ethics, equity, and other responsibilities |
|
X
|
|
LAB1
PRA1
|
fair working practices in the group and plagiarism consequences. |
|
3. INTERPERSONAL SKILLS: TEAMWORK AND COMMUNICATION | ||||||
3.1 Teamwork |
|
X
|
X
|
PRA1
|
work in project group, work distribution and leadership |
|
3.2 Communications |
|
|
X
|
PRA1
|
writing report and verbal defense of own solutions |
|
3.3 Communication in foreign languages |
|
|
X
|
PRA1
|
Course material in english |
|
4. CONCEIVING, DESIGNING, IMPLEMENTING AND OPERATING SYSTEMS IN THE ENTERPRISE, SOCIETAL AND ENVIRONMENTAL CONTEXT | ||||||
4.1 External, societal, and environmental context |
|
|
|
|||
4.2 Enterprise and business context |
|
|
|
|||
4.3 Conceiving, system engineering and management |
|
|
X
|
PRA1
|
Design choices to reach the required performance goals |
|
4.4 Designing |
|
X
|
X
|
PRA1
|
Design and Integration of a system with hardware IP-blocks in order to solve a given problem. |
|
4.5 Implementing |
|
|
X
|
PRA1
|
Implementing the design onto an FPGA |
|
4.6 Operating |
|
|
X
|
PRA1
|
Showing the correct functionality and performance of proposed design in real FPGA hardware. |
|
5. PLANNING, EXECUTION AND PRESENTATION OF RESEARCH DEVELOPMENT PROJECTS WITH RESPECT TO SCIENTIFIC AND SOCIETAL NEEDS AND REQUIREMENTS | ||||||
5.1 Societal conditions, including economic, social, and ecological aspects of sustainable development for knowledge development |
|
|
|
|||
5.2 Economic conditions for knowledge development |
|
|
|
|||
5.3 Identification of needs, structuring and planning of research or development projects |
|
|
|
|||
5.4 Execution of research or development projects |
|
|
|
|||
5.5 Presentation and evaluation of research or development projects |
|
|
|
This tab contains public material from the course room in Lisam. The information published here is not legally binding, such material can be found under the other tabs on this page.
There are no files available for this course.