Smart GPU Allocation¶

Client: Hugh Baker-Smith, Youron.

As AI models become smaller and more specialised, many training tasks no longer need data-centre-scale computing resources. Could AI itself help work out how best to train AI across GPUs distributed in different locations? Your task is to build a simulated environment representing a geographically distributed pool of identical GPUs and investigate how these resources can be allocated intelligently to maximise useful AI training within fixed time and financial constraints. Using the training of a small open-source AI model as a reference workload, the GPUs will have the same compute capability and hourly cost but differ in network latency, bandwidth and reliability, meaning that poor allocation decisions can waste both time and compute. You could explore approaches ranging from simple optimisation techniques through to machine learning or reinforcement learning, investigating whether more intelligent approaches can learn to make better decisions as conditions change. The challenge is to maximise training that successfully contributes to model improvement while minimising computation lost through communications overhead, GPU failures or interrupted jobs. Your simulator should allow different approaches to be tested, compared and visualised, showing how effectively they adapt to changing conditions.