Another M3 vs M4 Question/Thread

[christophermdia]

I know there are hundreds Probably of discussions on this, however, I want to think this through from an AI perspective. What do we all think the advancement of AI is going to be M4 forward? I was looking at an M3 Max however, I am not sure if i see an explosion of AI coming with rapid succession of M5, M6 etc … that puts todays devices behind at a rapid pace. While the M3 should still give me a good 3 years which is about my timeline, I dont want to fall too far behind. I currently have a M1 Pro, and it’s still working great, but about time to upgrade for the next 3 years. I could easily wait for the M4 but even then, I know i could continue this argument every year, but if AI is the future, want to keep up with the advances at a decent pace vs. falling too far behind. Not sure if this makes sense, but overall question here is what’s every prediction in how far/fast AI advances in the next 3 or so years and how they see the computing advancements from here on CPU vs. NPU vs. GPU?

 

[ChrisA]

If you need a new computer, buy one. Buy whatever is available.

If you don't need a new computer, don't buy one.

So, ask yourself if you really need a new computer. Is the "old" M1 somehow not able to do the things you need to get done? Perhaps there is some software you run that no longer works on M1, then you need to upgrade. Perhaps the M1 is just so slow that you are no longer productive because you find yourself getting up to get coffee while the old M1 processes your data?

Moving from M1 to M3 would meen that you can run benchmarks 25% or so faster. But normal work only uses far less then 100% of the CPU. Likey less then 50% most of the time. So a faster CPU means you will use enven less of it.

Want a car analogy? You have a "slow" car that can only go up to 100 MPH, But typically you can only drive 60 on the freeway. But you realy wnt that 200 MPH car. So you buy it to make you feel good. But still you can only drive 60 MPH.

Even the fastest computer still takes 20 minutes to watch a 20-minute cat video on YouTube. But on the other hand, perhaps you are a wedding videographer and you want to edit the ceremony footage shot on four cameras so you can show it in a loop at the reception. Your 13" M1 8/256 GB Macboor-air might not be up to the task. Time to upgrade because of the new business offering of "looped video at the reception".

Upgrade only what you can't do something in a reasonable time with the old computer. Or if you have enough disposable income to spend a few thousand dollars on a whim.

 

[Realityck]

I know there are hundreds Probably of discussions on this, however, I want to think this through from an AI perspective. What do we all think the advancement of AI is going to be M4 forward? I was looking at an M3 Max however, I am not sure if i see an explosion of AI coming with rapid succession of M5, M6 etc … that puts todays devices behind at a rapid pace. While the M3 should still give me a good 3 years which is about my timeline, I dont want to fall too far behind. I currently have a M1 Pro, and it’s still working great, but about time to upgrade for the next 3 years. I could easily wait for the M4 but even then, I know i could continue this argument every year, but if AI is the future, want to keep up with the advances at a decent pace vs. falling too far behind. Not sure if this makes sense, but overall question here is what’s every prediction in how far/fast AI advances in the next 3 or so years and how they see the computing advancements from here on CPU vs. NPU vs. GPU?

AI even though it seems it's coming fast is still in a rather infantile stage. There seems to be mistakes happening on a QA level to things not screened or tested before wider usage. The news is full of companies having AI working not quite the way they want.

So like all computer deployments, it's not a question of speed at this point in time, but waiting for software to mature and be well behaved without issues. Getting faster hardware is not going to make it easier for you until there are some processes that run too slowly compared to using a more up to date computer. Buying any hardware should be based on known computing usages that need to run at an acceptable speed to completion, not speculating about AI requiring faster processors. In other words wait until you see how stuff is running then decide if you need faster computers/devices.

 

[Chuckeee]

There is always something new in the next update.

 

[wilberforce]

There is always something new in the next update.

Yes, but advancements happen in spurts: a big advancement one year (like Intel>M1), followed by several years of smaller incremental advancements (like M1>M2>M3). Also, advancements in different areas often do not occur at the same time.
The key is to not miss (i.e., buy just before) a major advancement in the user's area of interest.
Personally, I think now is not a good time to buy if one's interest is AI. You will likely miss a major hardware advancement "spurt" for AI.

 

[Chuckeee]

Yes, but advancements happen in spurts: a big advancement one year (like Intel>M1), followed by several years of smaller incremental advancements (like M1>M2>M3). Also, advancements in different areas often do not occur at the same time.
The key is to not miss (i.e., buy just before) a major advancement in the user's area of interest.
Personally, I think now is not a good time to buy if one's interest is AI. You will likely miss a major hardware advancement "spurt" for AI.

But the M3 to M4 is still an ordinary hardware update, especially if your previous Mac is a Silicon Mac with more than 8GB. All of the AI “advancements” are software based and many are just part of the new Sequoia OS.

 

[mr_roboto]

Personally, I think now is not a good time to buy if one's interest is AI. You will likely miss a major hardware advancement "spurt" for AI.

Apple's major hardware advancement for so-called "AI" (it's not intelligence) was the A11 Bionic, the first Apple Silicon chip with a Neural Engine. That shipped in fall 2017, nearly 7 years ago.

M4's "AI" advancements are more hype than real, a concession by Apple execs that they need to be more visibly into "AI" so long as the bubble is bubbling. They changed the way they report Neural Engine compute throughput to roughly double the numbers relative to M3 and before, bringing the way they measure this throughput number more in line with what the PC industry is doing. However, if you measure the same way on M3 and before, those chip's numbers double too, and then M4 looks like a minor bump over M3.

Same goes for the "AI" improvements to the CPU - M3 and earlier had AMX, the direct ancestor to M4's SME. The only new thing is that now it's been formalized as an official Arm ISA extension so that Apple can expose it directly to third party software. However, Apple's CoreML framework should be able to use AMX on earlier CPUs, so it's still not as huge a step change as you might think.

The irony of Apple being painted as behind on "AI" until the 2024 WWDC is that Apple's actually been way ahead. They've followed a much more sober path than the bubble companies like OpenAI, building smaller features that are useful and reliable (Face ID, OCR so you can copy text out of images easily, classifying images in your Photos library, etc) rather than obsessing about building flashy but flawed BS and plagiarism generators like ChatGPT. (Speaking of which, it's disappointing on a moral level that they cut a deal with OpenAI at all, but at least it apparently involves not even paying OpenAI and should be very easy for Apple to back out of once the hype bubble pops.)

So, Apple doesn't need a revolutionary advance in on-device "AI" compute. What they've got is fine, and incremental improvements on it in each generation are fine. Unless you're all-in on the hype and think your plastic pal who's fun to be with is right around the corner, don't wait for something better. (If you are convinced of that, please consider becoming wiser.)

 

[theorist9]

A general comment that has nothing specifically to do with AI:

I respectfully disagree with the "if you need a new computer buy one, if you don't, then don't" advice often given on these forums, because I think it's too pat by half, and doesn't correspond to how things typically work.

Yes, there are cases where this binary situation applies—e.g., the application(s) you need to use will only run on a supported OS, and your computer will not be able to run a supported OS with the next OS release. Then you do actually need a new machine. [Yeah there's OpenCore, but...]

But it's rarely that binary. Instead, it's more typically: "My computer works, but the wait times are getting increasingly long as the software gets more complex. At what point will the reduction in wait times make up for having to spend the added money? And since I don't update my machine often, if I'm going to spend the money, I want the biggest reduction in wait times possible, so if there's a new machine coming out in six months, I might want to wait for that."

 

[ChrisA]

I know there are hundreds Probably of discussions on this, however, I want to think this through from an AI perspective. What do we all think the advancement of AI is going to be M4 forward? I was looking at an M3 Max however, I am not sure if i see an explosion of AI coming with rapid succession of M5, M6 etc … that puts todays devices behind at a rapid pace. While the M3 should still give me a good 3 years which is about my timeline, I dont want to fall too far behind. I currently have a M1 Pro, and it’s still working great, but about time to upgrade for the next 3 years. I could easily wait for the M4 but even then, I know i could continue this argument every year, but if AI is the future, want to keep up with the advances at a decent pace vs. falling too far behind. Not sure if this makes sense, but overall question here is what’s every prediction in how far/fast AI advances in the next 3 or so years and how they see the computing advancements from here on CPU vs. NPU vs. GPU?

Today I can run Meta's open source Llama3 (8b) model on my M2-Pro, 16GM Mac Mini at fairly good speed and still be running the norml desktop stuff at the same time.

Apple's "Open ELM" model is much less resource intensive then Llama3. I don't think youhave to worry, these user-side LLMs run fine on the M2-Pro.

I can't see why the required resourses will grow. a 3B or 8B model times the exact same amount of computation no matter how it was trained. Training will improves.

In all cases the larger models will NEVER run on your Mac, they will just get bigger. But thios is no different from today where the Google search engine can't possibly run on you Mac. You will need an Internet connection of some kind to access the bigger servers.

I think what lies ahead for on-device models is exactly what Apple diud when they released OpenELM, they found a way to redice the hardware requirements.

What I don't understand are the "good for 3 years" prediction. What data points did you use to find the slope of that line?

You have to figure that Apple is an iPhone company that has this side little side bussines where they sell Macs and watches and earphones. Everything thry do is designed to run in the iPhone and then if it can, the other products too. So to run this "Apple Intelegens" going forward, yu Mac only needs to be as powerfull as the last generation or so of iPhone. And of cource have an Internet connection for requests that can not be processed on-device.

 

[ChrisA]

But it's rarely that binary. Instead, it's more typically: "My computer works, but the wait times are getting increasingly long as the software gets more complex. At what point will the reduction in wait times make up for having to spend the added money? And since I don't update my machine often, if I'm going to spend the money, I want the biggest reduction in wait times possible, so if there's a new machine coming out in six months, I might want to wait for that."

It is "binary. At some point the wait times cross the treshold of reducing productivity. I remeber the days when (as a software developer) I would start a compile job and then get up to get coffee and which to working som other project. Then after some time check on progress. Those days are gone, unless you work in the field of AI.

Today it is very unlikey anyone is waiting even a few minutes for things they do frequently. OK, media trasncoding and video renders ca take a while but I'd guess not even 1% of Mac user even know that those words mean. All Apple Silicone based Macs are VERY good at normal office productivity work and web browsing.

Some people like to collect Fararris and other expensive sports cars. People do the same with Macs and iPhones. They do so because they can, not because they can't do their job without one. I wonder how many people buy the Mac Studio because they thing "what the hell, it is only a thousand buks more then an ungraded mac Mini" and for therm $1k is is their range of disposible income. It might evenbe the magority of Studios are sold this way, I don't know.

 

[Beau10]

In all cases the larger models will NEVER run on your Mac, they will just get bigger.

We're there already. Foundation models have been rapidly focusing on efficiency due to the costs of inference/electricity as these are unsustainable and the providers have been operating at a loss. And the trend to make things more efficient will continue, so there should be a convergence between cloud and consumer hardware.

An example of this in the open source space is the recently released Deepseek 2.0, a 230B MOE model with 22B activation (so it runs faster than Gemma2). It benches neck and neck w/ closed source foundation models, the only thing that decisively outstrips it is Sonnet 3.5. An M2 Ultra w/ 256gb can run it right now @ ~50-60 tok/sec. And w/ the M4 Max, we should have a laptop that can run it close to 40 tok/sec.

 

[ChrisA]

We're there already. Foundation models have been rapidly focusing on efficiency due to the costs of inference/electricity as these are unsustainable and the providers have been operating at a loss. And the trend to make things more efficient will continue, so there should be a convergence between cloud and consumer hardware.

An example of this in the open source space is the recently released Deepseek 2.0, a 230G MOE model with 22G activation (so it runs faster than Gemma2). It benches neck and neck w/ closed source foundation models, the only thing that decisively outstrips it is Sonnet 3.5. An M2 Ultra w/ 256gb can run it right now @ ~50-60 tok/sec. And w/ the M4 Max, we should have a laptop that can run it close to 40 tok/sec.

Yes, I agree, we should expect to see effecency improment. I am just now starting to look at quantum algorithms. Years ago this was science fiction but today you can sign up for Microsoft's "Azure Quantum".

I was a computer sciience grad student back when "classic AI" was the big deal (in the 1980s) and I've kept up with the field. LLMs were an astonishing development. "How could anything THAT simple work?" The answer is obvious in retospect. It all relies on compression. Then you force-fit 100 terabytes of "digital stuff" into a 400 gigabyte space all that is left is patterns and rules. AI is basically a search problem (finding the best compression) and quantum computing is VERY good at searching. An electron is literally in every place in the entire universe simulainiously, it makes looking "everywhere" faster by a few billion times. If you can figure out a ba-zillion details.

Today everyone is excited about LLMs but the next big thing will be quantum computing and this should have the effect of dramatically reducing energy requirements. Or maybe enabling many millions of times more compuputation with the same cost? In either case, we will not be having quantum computers on out desktops or inside or phones this century, and mayby not even in the next century. I have a lot of example code to go through and reading too but it looks like it is it.

​

 

[theorist9]

It is "binary. At some point the wait times cross the treshold of reducing productivity. I remeber the days when (as a software developer) I would start a compile job and then get up to get coffee and which to working som other project. Then after some time check on progress. Those days are gone, unless you work in the field of AI.

Today it is very unlikey anyone is waiting even a few minutes for things they do frequently. OK, media trasncoding and video renders ca take a while but I'd guess not even 1% of Mac user even know that those words mean. All Apple Silicone based Macs are VERY good at normal office productivity work and web browsing.

Some people like to collect Fararris and other expensive sports cars. People do the same with Macs and iPhones. They do so because they can, not because they can't do their job without one. I wonder how many people buy the Mac Studio because they thing "what the hell, it is only a thousand buks more then an ungraded mac Mini" and for therm $1k is is their range of disposible income. It might evenbe the magority of Studios are sold this way, I don't know.

I'm thinking more of repeated interactive tasks that are short enought that you wouldn't want to walk away from your computer, but long enough that waiting for them to complete is tiresome.

E.g., say you're doing some calculations in Mathematica. Often you will do a calculation, see the result, then based on that do another calculation. And there can be a whole string of those, multiplied by the normal trial-and-error. Depending on what you're doing, it's not uncommon for these to each take a few to several seconds each. So this is clearly not binary—there's no sharp threshold where if the calcs. average 6 s each your productivity is fine, but at 7 s each you require a new machine. It's a balance between the inconvenience of the wait time vs. the cost of getting a new machine, which is a gray area and a judgement call.

Likewise if you're someone who frequently needs to, say, convert image-based PDF's to text to make them searchable. Using Adobe Acrobat Pro's OCR with an M1-gen AS Mac, this takes about 0.5 s/page, so ~30 s for a 60-page document. Do you need to do these, and other similar tasks where the machine makes you wait several seconds, often enough to justify a new machine? As above, this is not binary; it's a gray area and a judgement call.

 

[ChrisA]

I'm thinking more of repeated interactive tasks that are short enought that you wouldn't want to walk away from your computer, but long enough that waiting for them to complete is tiresome.

E.g., say you're doing some calculations in Mathematica. Often you will do a calculation, see the result, then based on that do another calculation. And there can be a whole string of those, multiplied by the normal trial-and-error. Depending on what you're doing, it's not uncommon for these to each take a few to several seconds each. So this is clearly not binary—there's no sharp threshold where if the calcs. average 6 s each your productivity is fine, but at 7 s each you require a new machine. It's a balance between the inconvenience of the wait time vs. the cost of getting a new machine, which is a gray area and a judgement call.

Likewise if you're someone who frequently needs to, say, convert image-based PDF's to text to make them searchable. Using Adobe Acrobat Pro's OCR with an M1-gen AS Mac, this takes about 0.5 s/page, so ~30 s for a 60-page document. Do you need to do these, and other similar tasks where the machine makes you wait several seconds, often enough to justify a new machine? As above, this is not binary; it's a gray area and a judgement call.

This is rate. For this to happen the task you are doing has to be in this exact "Goldilocks" zone of "not too hard" but "not too easy".

What happens a lot when I do tsks like this is the computer is not the bopttlenext but my brain is. Mostly the computer waits while I think.

Look at the CPU uti;ization on Activity Monotor. It is is saying "80%" it means the CPU is not the bottleneck. For mpost tasks you never see all cores at 100%. Yes I can see it sometimes but itis VERY rare in normal use.

SO in the above case you describe a situation where it hits 100% intermitently for short bursts and maybe a faster CPU would reduce the length opf those bursts and save some milliseconds here and there adding up to minutes per 8 hour workday.

Let's say I bill clients at $2 per minute ($120 per hour). How many minutes to I need to gain before it is worth spening perhaps $4K or $6K on a new Mac Studio? What if I bill at $200/hour?

 

[Beau10]

SO in the above case you describe a situation where it hits 100% intermitently for short bursts and maybe a faster CPU would reduce the length opf those bursts and save some milliseconds here and there adding up to minutes per 8 hour workday.

Let's say I bill clients at $2 per minute ($120 per hour). How many minutes to I need to gain before it is worth spening perhaps $4K or $6K on a new Mac Studio? What if I bill at $200/hour?

There's more to it than simply that metric, these small interruptions of a few seconds here or there can impact flow - whether it's compiles, building packages, running tests, rebooting containers, little pauses in your IDE as it indexes, etc, etc. If your develop loop has a lot of these they really add up.

Several extra minutes a day in these types of pauses translates to way more frustration than actual time lost. They open up more opportunities to lose focus and goof off on something else if you're not super diligent. When actively pairing with other engineers it opens up more opportunities for side chit chat, and now whatever time wasted is multiplied - as someone in lead/staff/principal roles for a long time now, that's a major part of how I spent my time.

Couple years back I had a base 14" pro and base 16" pro. The 16" of course had roughly 20% more compute. Over a day, the difference was actually quite noticeable - and verifiable, a test suite on the 16" @ ~40 seconds took an additional 15 seconds on the 14". There were days where I'd take the 14" to a cafe for the morning then come back and finish it up on the 16", the latter just had more of an ease and reduced stress. And leaping from those to my 16" M3 Max is not quite doubling the impression, but it's a fairly palpable improvement.

As far as consulting goes, you also need to take into account tax breaks and recouping the cost from resale, so the cost to upgrade is roughly half or perhaps a bit less the actual cost of the new device.