-
Asked by game498huh on 4 Feb 2024.
-
Carl Peter Robinson answered on 4 Feb 2024: last edited 4 Feb 2024 2:46 pm
Cool question! I can provide a few crumbs regarding AI that might be considered interesting:
– Despite the enormous buzz and excitement surrounding AI these days, the field has been through two very difficult periods since its inception in the 1940s and 1950s. We call those periods “AI Winters”, where funding, enthusiasm, and the potential of AI were completely abandoned. During these winters, scientists and researchers had to wait for years to get another go at proving the viability of the AI theories and systems they were working on.
Do you think another AI Winter is coming? Or are we way beyond that ever happening again?– The breakthroughs in AI research and development since the late 2000s early 2010s came about because of a hardware microprocessor called a Graphics Processing Unit (GPU). The GPU was designed initially to help with graphics-related applications (e.g., 3D modelling software used in CAD) and… video games! Once scientists realised they could utilise the processing capabilities of GPUs, they were able to develop, train, and test their large neural networks, both practically and effectively. Using these large networks created a new subfield of machine learning called deep learning; ‘deep’ simply refers to the multiple number of layers in these large neural networks.
– Ever hear people talk about the number of parameters a large language model has, like ChatGPT? E.g., “The model has 175 billion parameters!” These parameters refer to the weights that get tuned in the model during the training process (i.e., the process where the model “learns” about its data). A weight is a fancy term for a number.
Just to get you thinking, imagine you were stood in a massive yard that stretched out so far you couldn’t see the other end. On the floor of the yard someone has laid out 175 billion small discs, in neat rows of 100 discs per row. They all have their red side facing up. The other side of each disc is coloured green. Now, imagine that in order to train that 175-billion parameter model, you had to turn over each of the 175 billion discs so that their green side faced up. Let’s say you’re super fast and can turn over one disc every 2 seconds. It would take you 350,000,000,000 seconds (175 billion x 2) to turn over all the discs. That’s the same as nearly 4,050,925 days, which is approximately 11,090 years!
That’s not quite how model parameters are tuned: they don’t all get set to the same value during training and there is a lot of parallel processing that goes on inside a GPU. But they are adjusted in some way and multiple times too, as the training process continues in cycles, to achieve the best results. Maybe not all of these weights are adjusted during each cycle, due to various methods employed to improve the model training process, but it’s still a heck of a lot of calculations to enable someone to ask ChatGPT, “How do I make the perfect cup of tea?”
-
-
Luke Humphrey answered on 5 Feb 2024:
Astronomy is a very interesting field right now. At university, a lot of my lecturers had spend time working in really remote locations because that’s where our best earth-based telescopes are located (to minimise light pollution from population centres – i.e. the darker it is, the better you can see the night sky). But they were very much the last generation of astronomers like this.
The best way to avoid light pollution is to send you telescopes into space itself and transfer the data back to earth digitally – and in recent years that’s what we’ve been doing. This means that astronomy is rapidly transforming as a profession from an observational science to essentially a data-driven science.
The scale of the cosmos is so unfathomably large that trying to process and make sense of all the data we get from satellite telescopes means astronomy is at the cutting edge of AI and data science techniques, which often spins out into other applications. I worked with something in my astronomy department who was working with a hospital to repurpose their astronomy code originally written to detect patterns in clusters of starts to detect patterns in clusters of cancer cells!
The other thing I find interesting is just the scale of data we have of galaxy observations. We’re now at a time where a single person could spend their whole lives looking through the images of galaxies returned from a single mission and literally never manage to see them all. This led to a cool moment in 2007 when a Dutch schoolteacher discovered a new type of astronomical object while participating in a project where the public could classify galaxies (which is a way to general the sort of labelled data you’d need to train an AI classifier).
It’s called Hanny’s Voorwerp, you can read more on the wikipedia article: https://en.wikipedia.org/wiki/Hanny's_Voorwerp
Comments