Fully Port DSS to ARM / Apple Silicon

This may be too obvious : 

User Story: As a Macintosh user who has enjoyed using DSS on my local computer, it would be great to use Dataiku DSS on a New Apple Silicon Computer. 

Value Proposition: 

  1. Having DSS working on Macintosh Computers has been an excellent entry point for folks interested in using DSS.  Installing DSS has been easy.  In two or three years Dataiku will lose this advantage of the ease of entry if it only supports Intel Computers. (With Version 11.0.1 this seems to be the case.) 
  2. Given the significant CPU & GPU performance gains in the first generation of the Apple M1 chip, it is likely that Apple’s Macintosh will be a performance leader in the laptop and desktop space for several years to come. However, to get those performance improvements DSS will need to support native Apple silicon (ARM64) instruction sets. 
  3. Give the custom Neural Network Processor in the Apple Silicon Chips it is likely that there will be development that can uniquely be done within the M1 Macintosh environment. 

 

--Tom
20 Comments
CoreyS
Dataiker Alumni

Hi @Baseltov I'm not really sure what you mean, the page hasn't been deleted. Here is the solution you are referencing:

https://community.dataiku.com/t5/Setup-Configuration/DSS-installed-on-Mac-M1/m-p/27513/highlight/tru... 

 

Looking for more resources to help you use Dataiku effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as ‘Accepted Solution’ to help others like you!

Hi @Baseltov I'm not really sure what you mean, the page hasn't been deleted. Here is the solution you are referencing:

https://community.dataiku.com/t5/Setup-Configuration/DSS-installed-on-Mac-M1/m-p/27513/highlight/tru... 

 

Baseltov
Level 2

@CoreyS That’s weird, I can’t access that page. If I click your link, or any link to that topic, I get the following Exception ID: 1938E724. And it does not open. Also if I manually search that topic name, it doesn’t show up in the search results either.

@CoreyS That’s weird, I can’t access that page. If I click your link, or any link to that topic, I get the following Exception ID: 1938E724. And it does not open. Also if I manually search that topic name, it doesn’t show up in the search results either.

CoreyS
Dataiker Alumni

Hi @Baseltov  sorry for the inconvenience you are facing. Despite testing across multiple accounts, unfortunately, I cannot replicate that behavior so my guess is that this may be a local issue. Can you try refreshing your browser page? Additional steps you can take include logging out and logging back in again, and finally clearing your cache on your browser. Hopefully, that does the trick.

In the meantime here is the solution:

Spoiler (Read more)

Hi there,
The Dataiku DSS Launcher works out of the box for Intel and M1 based Macs. No extra step is required for m1.

Best,
Alex

 

Looking for more resources to help you use Dataiku effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as ‘Accepted Solution’ to help others like you!

Hi @Baseltov  sorry for the inconvenience you are facing. Despite testing across multiple accounts, unfortunately, I cannot replicate that behavior so my guess is that this may be a local issue. Can you try refreshing your browser page? Additional steps you can take include logging out and logging back in again, and finally clearing your cache on your browser. Hopefully, that does the trick.

In the meantime here is the solution:

Spoiler (Read more)

Hi there,
The Dataiku DSS Launcher works out of the box for Intel and M1 based Macs. No extra step is required for m1.

Best,
Alex

 

@CoreyS ,

It is very cool that the new Macintosh launcher runs on an Apple silicon Mac.  I’ve found that 11.0.1 works better than 11.0.0.

The next step in this process is to move away from the Rosetta compatibility  mode and to native ARM64 code.  

--Tom

@CoreyS ,

It is very cool that the new Macintosh launcher runs on an Apple silicon Mac.  I’ve found that 11.0.1 works better than 11.0.0.

The next step in this process is to move away from the Rosetta compatibility  mode and to native ARM64 code.  

MichaelG
Community Manager
Community Manager
 
I hope I helped! Do you Know that if I was Useful to you or Did something Outstanding you can Show your appreciation by giving me a KUDOS?

Looking for more resources to help you use DSS effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as ‘Accepted Solution’ to help others like you!
Status changed to: In the Backlog
 
abdagon
Level 1

I just installed DSS on my MacBook Pro with M1 and Ventura, it worked flawlessly so far 😊.

However installing on ARM based Ubuntu did not (yet) work.

As DSS is coming to ARM architecture, what are the plans to make DSS available on ARM based Linux?

Supporting the statement of @natejgardner in this thread.

Data Driven Business Models | Digital Interoperability

I just installed DSS on my MacBook Pro with M1 and Ventura, it worked flawlessly so far 😊.

However installing on ARM based Ubuntu did not (yet) work.

As DSS is coming to ARM architecture, what are the plans to make DSS available on ARM based Linux?

Supporting the statement of @natejgardner in this thread.

@abdagon 

Welcome to the Dataiku community. We are so glad you have joined us.

I've been using DSS on Apple Silicon a lot recently. In most areas I agree it performs well.  (Thank you Dataiku Team.)

That said you may already know that the reason that DSS worked on Apple Silicon is due to Rosetta 2 Translation. This is a technology that comes on Mac that allows Apple Silicon ARM computers to run Intel code.

Although the Data Science Studio (DSS) code works.  I find in activity monitor that the architecture of the underlying code is still Intel not ARM.  Although Rosetta 2 is amazing technology, the use of it, tends to slow down performance by 20% or so. One can also not take advantage of some of the performance improvements that Apple Silicon provides in the area of GPU and other custom hardware.

I agree it would be great to see native support for Linux on ARM and Apple Silicon (based on ARM) with some support for the extended Architecture that Apple is providing.  This would allow data Scientists to work on individual workstation training models on local hardware and then promote their work with fewer hassles into production.

Now that it has been more than two years since this initial product idea.  How are others using Apple Silicon as part of your teams workflows?

--Tom

@abdagon 

Welcome to the Dataiku community. We are so glad you have joined us.

I've been using DSS on Apple Silicon a lot recently. In most areas I agree it performs well.  (Thank you Dataiku Team.)

That said you may already know that the reason that DSS worked on Apple Silicon is due to Rosetta 2 Translation. This is a technology that comes on Mac that allows Apple Silicon ARM computers to run Intel code.

Although the Data Science Studio (DSS) code works.  I find in activity monitor that the architecture of the underlying code is still Intel not ARM.  Although Rosetta 2 is amazing technology, the use of it, tends to slow down performance by 20% or so. One can also not take advantage of some of the performance improvements that Apple Silicon provides in the area of GPU and other custom hardware.

I agree it would be great to see native support for Linux on ARM and Apple Silicon (based on ARM) with some support for the extended Architecture that Apple is providing.  This would allow data Scientists to work on individual workstation training models on local hardware and then promote their work with fewer hassles into production.

Now that it has been more than two years since this initial product idea.  How are others using Apple Silicon as part of your teams workflows?

abdagon
Level 1

Thanks for the kind welcome message, @tgb417!

With a PhD in data management in the 80s, I am very keen to use the DSS features 😊.

I am well aware of the emulations (my first Apple Computer was a Mac II based on M680X0, then a PowerPC Mac, then Intel based notebooks and now a M1 notebook - I even wrote emulators for Z80 microprocessors a long time back 😉).

My request goes towards an ARM64 based hosting without an emulator such as Rosetta for an inexpensive «ever on» platform for prototyping, with a full license once going productive.

Reading through a few threads of this community I understand that the limitation is not so much about DSS, but more about underlying libraries such as Python, which must run natively. Based on Python information, 3.9 should be ARM64 native, with 3.11 being the most recent official version now. So it looks like it's mostly a question of porting DSS to a recent Python version.

Any dataiku statement would be much appreciated.

BTW - quiz question, who knows what ARM means? 😉

Data Driven Business Models | Digital Interoperability

Thanks for the kind welcome message, @tgb417!

With a PhD in data management in the 80s, I am very keen to use the DSS features 😊.

I am well aware of the emulations (my first Apple Computer was a Mac II based on M680X0, then a PowerPC Mac, then Intel based notebooks and now a M1 notebook - I even wrote emulators for Z80 microprocessors a long time back 😉).

My request goes towards an ARM64 based hosting without an emulator such as Rosetta for an inexpensive «ever on» platform for prototyping, with a full license once going productive.

Reading through a few threads of this community I understand that the limitation is not so much about DSS, but more about underlying libraries such as Python, which must run natively. Based on Python information, 3.9 should be ARM64 native, with 3.11 being the most recent official version now. So it looks like it's mostly a question of porting DSS to a recent Python version.

Any dataiku statement would be much appreciated.

BTW - quiz question, who knows what ARM means? 😉

While I would love to see Dataiku ported to ARM personally I think Dataiku will never port it to run natively on ARM. While there is interest by folks like us which use Apple Silicon Macs as computer and ML enthusiast there is very little penetration of Apple Silicon Macs in the enterprise. And we should understand that the majority of Dataiku licenses are bought by enterprises hence it is expected that these enterprise customers will drive the product roadmap. Though not officially supported DSS does currently run on Apple Silicon Macs in emulation in the same way that it runs on Linux VMs under Windows. I think this capability is enough for consultants, enthusiasts, etc to be able to use DSS for basic capabilities, demoes, etc. Any serious Dataiku user will probably want a more serious platform like a VM in the cloud or an on-prem server. 

While I would love to see Dataiku ported to ARM personally I think Dataiku will never port it to run natively on ARM. While there is interest by folks like us which use Apple Silicon Macs as computer and ML enthusiast there is very little penetration of Apple Silicon Macs in the enterprise. And we should understand that the majority of Dataiku licenses are bought by enterprises hence it is expected that these enterprise customers will drive the product roadmap. Though not officially supported DSS does currently run on Apple Silicon Macs in emulation in the same way that it runs on Linux VMs under Windows. I think this capability is enough for consultants, enthusiasts, etc to be able to use DSS for basic capabilities, demoes, etc. Any serious Dataiku user will probably want a more serious platform like a VM in the cloud or an on-prem server. 

@abdagon,

I to when through the Motorola 68000 to Power PC and Power PC to Intel conversions over the years. 

Regarding ARM.

Spoiler (Read more)
If I understand correctly.  Back in the day it meant:

Acorn RISC Machine

Because the British maker of the BBC Micro and the Electron Computers Acorn Computer created the first ARM processor.

Today it seems to be morphed into.

Advanced RISC Machine

And Apparently the folks at ARM don't even use it much as an acronym. 


The value proposition for enterprise is really around the problems that Intel is having at making innovative performant but low power chips.  Electricity and cooling cost real money. However, @Turribeach .  I take your point.

--Tom

@abdagon,

I to when through the Motorola 68000 to Power PC and Power PC to Intel conversions over the years. 

Regarding ARM.

Spoiler (Read more)
If I understand correctly.  Back in the day it meant:

Acorn RISC Machine

Because the British maker of the BBC Micro and the Electron Computers Acorn Computer created the first ARM processor.

Today it seems to be morphed into.

Advanced RISC Machine

And Apparently the folks at ARM don't even use it much as an acronym. 


The value proposition for enterprise is really around the problems that Intel is having at making innovative performant but low power chips.  Electricity and cooling cost real money. However, @Turribeach .  I take your point.