Dev Talks: Native Access Startup Journey and Next steps
Hi everyone.
As promised, here is our quarterly update of Q1 and what’s coming in the next quarter. I’ll start with a topic deep dive, and go into what we’ve done and will do next afterwards. I’m eager to hear feedback on this format.
The Startup Problem
Since we launched we’ve been tackling many issues, the main topic being long startup times. So why was Native Access such a pain to start up, and why did it take so long to improve that?
The core problem lies with the tasks that we were executing. As mentioned in our previous Devtalks, the new Dark-themed Native Access (NA2) and the NTK Daemon are two separate entities working together, and that makes it so we can iterate faster by tackling problems in each entity independently. But when it came to startup times, we were very dependent on each other to go through the following steps:
- Install NA2
- Have NA2 install the NTK Daemon
- Connect NA2 to the user’s Native ID (log in, if the user isn’t already logged in)
- Retrieve all the product details from our Content Management System (CMS)
- Retrieve the logged-in account’s licenses and subscriptions (or known products)
- Activate the user’s products
- Check the status of the user’s products, whether they’re installed, broken, etc.
- Do some filtering to display what’s relevant for the user (i.e. filter out bundle licenses)
All these steps are still necessary today, and initially we had these tasks executed in sequential order. This excludes additional tasks, such as setting up the helper tool for system permissions to install products, or asking to install Rosetta 2, just to name a few. While we coded these steps above, we already experienced extensive startup times. Though we knew we’d want to keep tackling this, we couldn’t hold off on launching the other improvements in NA2 any longer.
After we launched NA2 with version 2.1.0, we were well aware of the long start up times and issues within each phase of the startup process. We definitely underestimated the user impact though, as our teams were experiencing a max of 30 seconds, while users would be experiencing up to a few minutes, regularly. To get an idea of where we started near launch, we had the following time ranges:
- 0-10s startup times (30%)
- 10-30s startup times (54%)
- 30-60s startup times (11%)
- 60s+ startup times (5%)
September 27, 2022
In Q3 2022, we stabilized many parts of NA2. With release 2.6.0 we adjusted the activation system, skipping already activated products. Although a minor improvement, we still expected a slight shift in our metrics, but at the time there was one factor we didn’t consider: overall traffic. As shown, between 20 September and 29 September, we went from:
- 0-10s (21%)
- 10-30s (57%)
- 30-60s (15%)
- 60s+ (7%)
to:
- 0-10s (18%)
- 10-30s (59%)
- 30-60s (14%)
- 60s+ (9%)
The worse startup times were due to an increase in overall traffic, from 14.8k to 21.2k daily active users. But in the meantime, the NA2 team was already busy completely overhauling the way the interface interacts with state changes.
November 21, 2022
Internally, we were excited about the things coming in 3.0.0: changing the entire architecture to respond to backend events reactively as opposed to proactively, thereby not blocking the app in all of the steps mentioned above (mainly post login) and being much more performant. This took us all summer to complete, as we found many opportunities to stabilize and improve other parts of the app along the way.
While we definitely realized the performance upgrades, something was still holding back the expected start time improvements. On this day, we experienced a large influx of users following the annual cyber sales period, and our backend servers were overloaded. The work to bring them back online and stabilize them coincidentally led to improvements in the startup times. Calls were taking less long and our app was reacting to them way more efficiently, leading to the following changes overnight:
- 0-10s (54%, up from 13.6%)
- 10-30s (39%, down from 69%)
- 30-60s (3%, down from 13%)
- 60s+ (2%, down from 3%)
These trends continued. We felt like we were out of the danger zone, but still had work to do to achieve 75% of startups taking 0-10s.
February 20, 2023
In Q4, we collaborated across departments to generally overhaul the product activation flow to further improve startup time. The problem was that we were checking if we needed to activate all of the user’s products every single startup, which takes a lot of time. We tackled some of this earlier on NA2’s side by skipping some products that were already activated, but it needed more support from more parts of the company to have a larger impact, as well as just activating products with updated statuses as opposed to checking if every product needed activating. While activating all products makes sense for the first time startup, every startup after that we only care about account or status updates, such as subscriptions running out, new products, or transferred products. But even those, we could do when they happen rather than check on startup, and even more so, we could move the moment we activate over to when we install a product as well. All these points culminated in release 3.2.0, which improved startup times significantly:
- 0-10s (80%)
- 10-30s (16%)
- 30-60s (2%)
- 60s+ (2%)
This was beyond our goal of 75% of startups taking 10 seconds or less! We recognize that Daemon installation times can be further improved, but we still expect further improvements as more people use version 3.2.0. As of releasing this post, we’re sitting at 85% of all startup times in the 0-10s time window.
So what now?
We’re extremely excited about the startup time improvements. We have a few more steps that we’re getting to here and there, like mitigating the amount of times users startup Native Access and need to install dependencies again, and we’re also aware of some users still experiencing bad startup experiences. But we’re beginning to focus on other topics.
We can still make large improvements to start up times through a new initiative: Daemon Stabilization. We recognize that the Daemon is crashing more often than we’d like, affecting processes such as startup and installations. To get a better understanding why these crashes occur, we’ll work on improving our diagnostic systems, which will help us further our needs to improve download success rates, have better app session performances, and a more solid startup experience.
Another thing we’ve decided to bump up in priority is stabilizing the download queue. Our current problem is that NA2 and the NTK Daemon are sharing bits of the functionality, which leads to inconsistencies in our diagnostics tracking, downloads disappearing when closing an NA2 session, as well as creating difficulties in testing and quality assurance. We will move this functionality completely into the Daemon, while also addressing bugs and making minor improvements, such as retrying failed deployments in cases where the network connection was lost. So far, we’ve found several major improvements we can make, and are excited to ship this in Q2.
But even more exciting is the fact that we finalized UX discussions about uninstall, which means work can get started. For Q2, we intend to give you the ability to uninstall products, starting with content products and progressing through all other types of products.
And lastly, I’m loving the response to our transparency efforts! To make sure you’re in the loop for all the exciting things happening in Native Access, here’s a preview of the current initiatives we’re working on:
Note: The order of topics and priorities are not fully locked in and subject to change. Everything in progress is in active development. No accurate time frame can be given at this stage.
And of course, a reminder that our team reads all incoming feedback, so please do drop a feature request here if you have one. For any support you might need, please visit our support channel and drop a ticket there. The support team has been helping us tremendously by keeping an eye on emerging concerns, frustrations, and negative trends, and we couldn’t do all that we’re doing without them.
Conclusions
Native Access 2 is fulfilling its promise to be more transparent. This quarter we’ve given you a glimpse into our startup issues, and next quarter we’ll talk more about how we’ve been tackling our shortcomings with deployment success rate.
A few important things: with 3.3.0, we deprecated Rosetta 2 and officially dropped support for Mac OS X 10.14. You should be able to use Native Access still, but for any issues you might face we’ll implore you to update your operating system first. Additionally, add serial messaging is now more complete, so you’ll get more clear feedback if something goes wrong, and the ability to take action. Please look out for our release notes of any bug fixes that might solve some concerns you are facing, as we’re hard at work trying to make things better.
Eager to see your feedback and discussions down below. Really enjoyed it last time!
Comments
-
Hayo will be happy to answer any questions you have. Be sure to drop your questions here before this thread is closed on Thursday, April 20!
0 -
Why even closing it?
1 -
Ok. I’ll leave my thoughts here
Sorry if I will sound a little critic (I always do 😂) but I’m deeply convinced that criticism is the key to improvement.
First of all, THANK YOU very much for this devtalk full of info and that sounds quite honest. It’s a breath of fresh air that is really needed in NI.
Now to the critic: my post here is more of a question to the other users (just to know if I’m the only one thinking this or if is something more widespread into NI users community)
I see a lot of work has been putted into this, but what I see is a deep focus on “startup times”, “let’s make this faster”, “let’s improve the things under the hood”.
Now… maybe I’m the only one thinking this…but Native Access is a program we use now and then, just when we need to install something new or update something already installed. Not something I need every single day and that improve my musical experience.
And therefore having something that opens with my computer startup and runs constantly in the background (sometimes causing also troubles) only for the few times I need it is something I’m not too happy with.
That said, I find quite strange the main focus of the developers is all about “making Native Access start in under 10 seconds”. I know nowadays world is all about being efficient and fast as lightnings, but, talking for my case, I’m not too concerned about having to wait a little more for a program that I use once in a month to open.
Being active on the forum to help other users quite a lot lately, what I see is many people having troubles installing their softwares (being it for concrete problems with the installation software or for lack of experience with this world) and questions asked but only partially answered (or even almost ignored). Problems that after a long time have not been solved, making people more and more frustrated with their experience with NI.
Therefore…what I’m saying is that you are talking about months and months spent improving a startup time by 10-15 seconds, while maybe all this time could have been used to be sure the many installation problems will not happen again, or even to come in and talk to the people having this problems to try to find a solution (or even only not making them feel abandoned). Or to listen earlier to the requests that users are making from a long time and that you are addressing now, following the initiatives you showed us.
So…in the end this is more a “suggestions for the future” thing more than anything else. What has been done, has been done. And we are grateful also for this. Thank you for your work!
And once again, REALLY THANK YOU for finally applying some of the transparency that lately has been waved but that sometimes is still…quite hidden and obscure.
Now my last words for the other users: what do you think? Is THAT important for you that a program that we use now and then is lightning fast to open or you care more about stability, certainty of working installations, help (this yes lightning fast!) if something goes wrong and feeling supported by the brand we invested our money and faith into and we elected as “our partner in music making”?
I’ll read your answers with attention, and if I find out that most of the users have different priorities than mines, I’ll shut up (at least on this subject 😂)
Thank you for listening and… to the developers: good work…to the users: good music making 👍🏼
4 -
I suspect that this emphasis on startup times might indicate that NI anticipates in the future, many users will be using Native Access more often than in the past.
Why this would be is an interesting question.
1 -
Hi All. Long-time lurker and first-time poster here.
I've been waiting some time for Rosetta 2 deprecation and have attempted to install 3.3.0 on my MacBook Pro 14 running Ventura 13.3.1. My problem is that NA2 fails to install with the "Unable to start the NTK Daemon" message.
I'm guessing that NA2 is failing to install the NTK Daemon as the installer for this within the NA2 contents prompts for Rosetta 2 to be installed when run separately.
Is this intended behaviour for the present moment or should both NA2 and the NTK Daemon install natively on Apple Silicon without Rosetta 2 support?
Thanks
1 -
100% agree with all of LIF's post.
Native Access is a (minor) utility (at best) that for most of us ranks right up there with launching Device Manager or something once every week or two.
While I do understand a need to have NA2 perform well - when needed - this thing seems woefully complicated and unnecessarily complex when compared to your other peers in this space - like say Product Manager from Toontrack, Arturia's Software Center or even Waves Central.
All of these tools run smooth, fast and take care of business instantly while not littering a users machine with extra services etc while doing it.
Your team might consider focusing on simplifying NA with a renewed focus on ensuring just two areas that matter to us - are bulletproof:
- A single app that launches when needed and closes down permanently when not (NO Services)
- Install (Activate) without fail and then show the users existing product lineup.
Extras could include a bulletproof Locate (or better yet - a SCAN and Locate) and a clean, deep uninstaller as needed.
A super wish list item of mine (that I do not think anyone has mentioned) is an ability to back up my entire NA Profile (products, licensing/activations, file paths/locations etc) and be able to import this BACK into NA when I rebuild my DAW. If designed right along with an optimized Scan and Locate - one could have a restored NI profile up and rolling in no time (if one leaves all the major file structures intact on an existing drive - as many of us do in a rebuild scenario).
All these other guys have a product manager that does all of the above (except backup) with no muss or fuss. It's time NI joined that party.
VP
4 -
As I implied in my last post.. the "answer" to this "odd" emphasis on Native Access's performance might very will be because of it's crucial role in something in the future. Hmmm?
0 -
Valid criticism. Loads to unpack so I'll try to split it up into two posts for easy reading.
This edition was definitely a deep dive with emphasis on startup. Last one's was why a Native Access 2, and the next one we'll be doing will be about installation success, as it's something we're putting at priority level 1 now and doing a LOT of stuff on. If I DevTalked about EVERYTHING, I'd be writing a novel, so we're breaking things down bit by bit. Hopefully that's okay with everyone for now.
Our team is able to work on multiple things at once, so it's not like we dedicated 100% of our resources on fixing startup times this whole time. But we try to tackle one or two major topics at the same time. Installation success has been high on our list, but admittedly not high enough as it should've been, for which I take responsibility. The issue is that we do want our users from A to B quickly, and the fact that Native Access startup was not just unstable but unnecessarily slow (like, 1 minute each startup slow) were hurdles we were trying to overcome. It was definitely a top contender for the things users rightfully frequently complained about.
At the moment installation success is priority one. I'll go more in depth next quarter on everything, but we spent Q4 of last year on a lot of research and telemetry gathering through our diagnostic tools to see how deployments were making it through our funnel. We noticed a lot of deployments weren't making it through the funnel at all, not even showing an error, and just disappeared. This didn't just happen with active deployments though, also with queued ones.
Daemon instability is one reason that downloads disappear, but others include that Native Access and the Daemon sharing functionality for the download queue was great for shipping quickly, but bad for testability and reliability, and made for a difficult communication challenge between the two services. We're moving all that to the NTK Daemon. We've found a bunch of areas of improvements along the way, like how we react to network connectivity dips or certain types of errors, and we plan to ship that along with the migration work we're doing.
We're still doing plenty of research into how we can make the experience more reliable, so loads of stuff happening on this topic. Though, just to note how we haven't simply forgone this topic until now, we did resolve a few bugs that made it so that downloads we're completing more often, a big one being responsible for 10% of all our deployments simply not making it through the funnel. It might not be very noticeable for the userbase, but it's definitely been noticeable for us.
1 -
Second part to do with lack of transparency:
This is definitely something I realized as last year went on, is that as the PM of these two services I did not do enough to communicate with the user base. I've since launched the quarterly DevTalks, and am showing my face more often in the forum than before. I hear you though that I could do better still, so I appreciate the feedback!
However, we can't tackle everything at once. As we dive deeper into each topic we need to address, we keep finding areas of improvement that we address while we tackle the topic as opposed to immediately ship a fix. I enable this, because while we work on these larger topics and find these smaller issues we can resolve along the way, I don't want the team to be doing double work, fix now, then adapt that same fix into the refactor later. The sacrifice here is longer ship times, but the benefit is that the fixes we make are more rigid. I ask a lot of patience from you all, and for that I do apologize.
1 -
I'm not sure what the issue people have with NTK Deamon running in the background - I've not been aware of it causing me any issues (granted I'm not sure what it does most of the time).
Given the number of complaints I've seen, I don't think it's unwise to have done the work on startup times. I appreciate not having to wait around for things to load wondering if they're stuck.
I'm impressed with the amount of transparency NI is working - who else does this? Good to see uninstaller and locate all on the roadmap on top of performance and reliability things.
🙂
There was mention of spending less time on improving startup times and instead addressing people's issues on the forum (or at least how I interpreted), but that's pretty clearly two different departments. Some issues I've seen here and on reddit seem to be account level issues that they should be contacting support about. That said I've also seen an increase in mentions of slow (weeks if at all) support response times recently.
0 -
Wow…even more in depth explaining. I’m overwhelmed! ☺️ Thank you
Just a quick response.
First: believe me, the transparency critic wasn’t directed to your department. There are others doing quite worst 😉(but they are improving too. Slooooowly, but improving)
Second: (always a matter of point of view, and probably yours is more authoritative/prestigious being this your work) I’m not so sure quicker singular fixes aren’t the best solution. I do understand the “trying not to make the work twice” point, but sometimes having to wait 6 month or more for a fix to a problem is quite long (at least for the important and invalidating ones). Some people in this “music making” world make this as an hobby and maybe do it for a couple of years or so. If something not functioning will be not functioning for half the time of their experience, it makes of it a not enjoyable experience.
In my opinion quicker smaller fixes are a better way. But as I said, I’m not in your position to judge what’s better
0 -
Great work! Thanks for sharing a roadmap, it's very refreshing and I wish this could transpire with other products, some really need it 😉
I got a few lighthearted questions that don't require a super serious answer, if anything is too personal feel free to ignore it.
- Now that NA is more mature, any regrets about choosing Electron VS other approaches?
- Is the ability to select which plugins we want to be installed (VST2, VST3, AU, AAX) being considered?
- As a product manager, how is most of your work time spent? Actually coding or planning, coordinating, making decisions, etc?
- Any chance the text inside NA could be selectable? So we can copy it, especially the release notes. (Sorry I don't want to exploit the QnA to ask for things but this sounds sooo simple 🙏)
1 -
Ooooh yeah…plug-in types installation choice…I almost forgot about this possibility we had till not long ago 👍🏼
0 -
Not entirely. We want our users in our products and hoping we can help make that a smooth transition. That being said, I do believe that, since almost all of our customers need to go through Native Access in the beginning, we could contribute to the onboarding experience of our userbase to our ecosystem, our products, and the act of making music. I wouldn't be opposed to exploring those options. Whether there's anything in the works to pivot Native Access to that at the moment: our focus is on making sure Native Access does its current job greatly and nothing else. We want to make installing, getting up and running, and managing your products a seamless experience, through maintenance on our existing feature set and building features to complete the product management toolkit for our customers. The stack we have now keeps our options open for how we can do this.
2 -
We've deprecated Rosetta 2 support and removed all instances of it from our codebase, so this shouldn't be the reason you can't get set up. Highly recommend you reach out to our support team. They know better than I do what might be causing this. Apologies for the inconvenience
0
Categories
- All Categories
- 19 Welcome
- 1.4K Hangout
- 60 NI News
- 735 Tech Talks
- 3.9K Native Access
- 15.9K Komplete
- 1.9K Komplete General
- 4.1K Komplete Kontrol
- 5.5K Kontakt
- 1.5K Reaktor
- 364 Battery 4
- 817 Guitar Rig & FX
- 416 Massive X & Synths
- 1.2K Other Software & Hardware
- 5.5K Maschine
- 7K Traktor
- 7K Traktor Software & Hardware
- Check out everything you can do
- Create an account
- See member benefits
- Answer questions
- Ask the community
- See product news
- Connect with creators