A review of the UK’s large-scale computing ecosystem and the interdependency of hardware, software and skills. This report provides evidence to support the establishment of a coordination function and sets out the key policy areas to consider.
- Key concepts in large-scale computing
- Users and uses of large-scale computing
- The global stage, data and hardware
- UK infrastructure, software and skills
- Achieving the UK’s potential in large-scale computing
Key Facts
Top 500
The United Kingdom currently has 12 systems in the Top500 lists of global high-performance computing systems. The 12 systems represent 2.4% of global installations and 1.4% of total performance capacity.
Cloud computing
In 2018, 74% of high-performance computing sites were found to run some of their workloads in the cloud.
New use cases
New uses of large-scale computing including smart cities, emergency preparedness and resilience, energy networks, public health and digital twins.
The UK’s highest-ranking machine
The Met Office supercomputer is the UK’s highest-ranking machine at #37.
Skills in large-scale computing
The UK has the largest software industry in Europe. The UK software industry contributed to direct value-added GDP of £70.3 billion in 2016. It directly employs nearly 700,000 workers.
Glossary of Terms
See Appendix A for full glossary of terms.
| Term | Meaning |
|---|---|
| Access models | A method by which users can access the capability of a computing system (for example, cloud computing) |
| Algorithms | A set of instructions that, when given to a computer, are used to solve problems |
| Artificial intelligence | A broad area of computer science concerned with the use of computer systems to carry out functions analogous to human cognition |
| Cloud computing | A rapidly emerging access model where computing infrastructure is accessed on-demand via the internet |
| Exascale | A high-performance computing system that is capable of at least one Exaflop per second (a system that can perform more than 10 floating point operations per second) |
| Large-scale computing | Computer systems where processing power, memory, data storage and networks are assembled at scale to tackle computational tasks beyond the capabilities of everyday computers. Often involves the widespread use of parallelisation (see Appendix A). An umbrella term encompassing terms such as high-performance computing, high-throughput computing, supercomputing and novel computing paradigms |
| Petascale | A high-performance computing system that is capable of at least one Petaflop per second ( a system that can perform more than 10 floating point operations per second) |
| Quantum computing | An experimental form of computing which utilises the probabilistic qualities of quantum mechanics. It has the potential to massively accelerate certain computational tasks (also see Appendix C) |
Preface
Large-scale computing (LSC) has revolutionised our lives in fields of national importance such as weather and climate modelling, and financial services. It is an important enabler of R&D as demonstrated by DeepMind’s remarkable recent breakthrough on protein folding. Computing power also underpins key technologies such as Machine Learning and Digital Twins.
The pace of innovation in LSC is accelerating rapidly, as illustrated by the fact that the highest performing system today is 175,000 times more powerful than the largest system 20 years ago. This brings with it new opportunities and emerging applications in areas such as fusion energy, energy networks, digital twinning, and public health. With new opportunities come challenges that the UK must address if it is to realise the true potential of LSC and the Prime Minister’s ambitions for the UK as a Science Superpower.
This report describes the LSC landscape in the UK and sets out the building blocks required to create a world-class computing ecosystem. While quantum computing is likely to be of critical importance, we are some years from commercialisation and therefore it is not covered in much detail. The core of this report is based around seven challenge areas and options to overcome each: the need for better national coordination; nurturing a healthy UK ecosystem; ensuring we have the right mix of hardware, skills, and software; minimising energy consumption; and the UK supply chain.
LSC forms part of the UK’s national infrastructure and many of the key opportunities and challenges span multiple sectors. The Government therefore has a significant role to play in nurturing and supporting the UK ecosystem both as a consumer and a funder. Therefore, our primary recommendation is for the establishment of a team within Government to take policy responsibility for large-scale computing and address the challenges that are identified in the report. Industry, as a major user and innovator of large-scale computing, will be part of the solution and need strong ties to the coordination team within Government.
This report has been prepared through interviews with officials, experts and industry stakeholders from across the ecosystem under the direction of a steering group comprised of departmental Chief Scientific Advisors. I would like to thank them all for their invaluable contributions.
Sir Patrick Vallance
Government Chief Scientific Adviser
Executive summary
There is a strong case for continued public investment in large-scale computing. However, this needs to be coupled with improved long-term strategic planning, driven by close collaboration between Government, academia, and industry.
Large-scale computing is an essential tool for solving industrial and scientific problems. It is used to analyse highly complex or data-intensive problems involving simulation, optimisation, data processing and artificial intelligence (AI). Large-scale computing supports a range of R&D-intensive sectors and is used as a research tool at many of the frontiers of science. Several large-scale computing systems, such as those used for weather forecasting, form part of the UK’s critical national infrastructure. Computing needs vary substantially across sectors, with users having different requirements in terms of both capability and support requirements:
in the public sector, service reliability is a key consideration. For public sector users, cybersecurity and the physical locations of infrastructure are other key considerations
academia requires access to systems of a range of sizes and architectures owing to the diversity of programs run
industrial users of large-scale computing use a wide range of access models. Private sector use of public systems often includes a requirement to publish results and cybersecurity is a key concern
small and medium enterprises (SMEs) often do not have an awareness of the range of business problems that large-scale computing can solve or the technical requirements to use it. In addition, SMEs often require support in adapting their software to run on large systems
The UK is a global leader in a number of computing domains, including in areas of software development, computational modelling, data analytics, cybersecurity, AI and machine learning. [footnote 1], [footnote 2] However, the UK’s large-scale computing infrastructure lags behind other major global economies and would benefit from a refreshed, longer term focus that takes into account interdependencies such as skills and software. As of November 2020, China and the US had 214 and 113 of the top 500 computer systems globally, and France and Germany had 18 and 17. In contrast, the UK had 12. [footnote 3]. The UK’s share of global high-performance computing capacity has decreased by three-fifths over five years, falling to 2.0% in 2019. [footnote 4]
Large-scale computing is a fast-changing field, where systems have a working life of five to eight years before becoming obsolete. We are entering an era of unprecedented hardware diversification, driven by the development of new computing architectures for specific tasks, in particular for AI research. Advances in data science have also driven demand for large-scale computing. As a society, we are generating far more data than ever before, and without the means to analyse it, its value is left unrealised.
Cloud computing has become an increasingly popular access model. Cloud access provides opportunities for new users to reap the benefits of many (but not all) forms of large-scale computing by offsetting upfront costs with cost of use. Users can assess their needs and choose a cost model to suit, whether that be on-site hosting or access via the cloud. Commercial cloud service providers are increasing their range of capabilities. This is still a nascent sector, and the market is currently dominated by a limited number of leading providers based in the US.
There is a strong case for continued public investment in large-scale computing. Access to world-class computing capabilities enhances UK competitiveness across a number of sectors. Lack of sufficient access to computing can create bottlenecks within research, development, prototyping and testing. The cost of leading-edge systems is greater than any single institution or business can afford. The UK Research and Development Roadmap describes supercomputing as a key part of our digital research infrastructure. [footnote 5]
Large-scale computing plays an essential role underpinning UK R&D, helping to achieve ambitious ‘moonshot’ challenges, such as zero emission air travel. [footnote 6]
Investment in computer hardware alone will not be sufficient. To achieve the full potential of large-scale computing and allow the UK to compete at an international level, longer-term strategic planning is needed. Better coordination is required between Government and the publicly and privately funded research sectors. This report focusses on the functions that will be necessary for the UK to address these challenges and seize on the opportunities brought by large-scale computing.
Global trends spark important strategic questions about the types of national investments the UK should make. Our vision is to establish a dedicated oversight group tasked with providing effective coordination to the large-scale computing ecosystem and its users, to level up Britain as a science superpower.
Recommendations
1. National coordination. At present, there is no overall national roadmap for large-scale computing, and there is no single team within Government that provides coordination or carries out wider strategic assessment of UK needs. This has resulted in a wide variety of different procurement practices across departments.
- We recommend establishing a team within Government to provide policy leadership of large-scale computing. This team would be responsible for developing a rolling long-term roadmap for large-scale computing. This roadmap should cover the whole UK computing ecosystem, including software, skills, and user needs. This would help to improve resource sharing between organisations and provide a conduit for industry engagement. Finally, this team could advise Government on computing procurement, helping the public sector become a more intelligent customer.
2. Future systems Access to leading-edge computing is vital for many academic and industrial fields. Future computing capabilities will open up new application areas and help to solve problems that are currently intractable. Projects are underway around the world to develop exascale computing which would deliver systems roughly 140 times faster than the UK’s fastest system as of November 2020. These systems could mark a step-change in certain capabilities. Meanwhile, novel computing architectures could greatly…
Large-scale computing: the case for greater UK coordination (HTML version)