Using a high performance NAS/file server for libraries - have you done this?

Milkman · 2022-01-27T22:30:59+00:00

There was an error rendering this rich post.

Hello! My name on the forums before the reset was Jeremy.

I am about to buy and build a Synology NAS, such as the DiskStation DS420+, or the larger DS1621xs+. I'm not 100% on the model yet.

A NAS/fileserver of this capability should be able to host K13 and other libraries across a switched 1000mbit wired connection without any problem, and I have a robust WAN/LAN router/security device. I am the network/security engineer at home, and as such we don't have any network config or performance issues that we would be dropping this NAS into.

Does anyone have any experience either with Synology NAS, other branded NAS, or their own JBOD sort of NAS for storing libraries and samples for K13 + other libraries? If so, what was your experience like over time? Did you have any trouble with this config?

I've never stored any libraries or any of my music data anywhere but the local workstation, so this will be my first. Your experience on this would help! : ) Thanks

EDIT: forgot to mention -- there will be high performance (7200rpm, 4-6Gb/sec) drives inside.

Find more posts tagged with

Komplete Standard

Comments

Jeremy_NI

https://community.native-instruments.com/discussion/566/using-a-high-performance-nas-file-server-for-libraries-have-you-done-this

NAS drives are not supported. A lot of issues were reported, we would definitely recommend them for our products.

Kubrak

Generally, one should not install NI SW on network. According to Licence...

I do not have NAS experience. People reported, it works OK. Your NAS seems to be strong. I guess, you would have to map NAS as LAN disc. And maybe even set something in registry so that it is not reported as LAN disc... Not sure, it has been discussed in old forum, what to do....

But still.... I have internal NVME disc (Samsung SSD 980) that reads 3500 MB/s, which is roughly 30x more than LAN 1 Gb connection.... And loading bigger projects takes easily 10-30 seconds. It took much, much longer with internal SATA6 SSD (600MB/s at max). And LAN is yet considerably slower... So, for me no way, I could drink few cups of tea while openning single project...

So, NAS solution might work for you or might not. Depending on, what kind of NI content you use.

While I do have a NAS, I went quite oposite route. Internal HDD -> Internal SATA6 SSD -> Internal NVME SSD. And next time I will buy even faster NVME SSD.

Milkman

https://community.native-instruments.com/discussion/comment/3701#Comment_3701

Thanks for the reply. 30 year net/sysadmin here.

The big issue for me here is --- I dont recall where in my NI licensing it claims I cannot use network attached, or indeed local USB or eSata storage, which should be viewed similarly from a licensing and legal perspective. This is my home network, I manage my own local storage to my specifications, and if a fully mapped network drive is not suitable per licensing and NA can't "see" it, this will be a very frustrating outcome. I will have to investigate this right away, as this will be a major non-technical issue for me if so.

There is no real performance bottleneck(for this purpose) on a fully switched network with a 10Gb backplane, I have 32GB of system memory, the NAS will have m.2 buffering, and I can't imagine accessing libraries across that connection would be a problem, even with some milliseconds of additional latency vs local spinning disks or SSDs. I dont need my sample libraries to load in .02ms, and if it takes 300ms, I am entirely ok with that.

The licensing is the big problem here, if so. I will investigate ASAP.

Milkman

https://community.native-instruments.com/discussion/comment/3646#Comment_3646

Arrrgh, sorry, I read the thread in the wrong order. The new forum system tripped me up. =P

Ok, so NAS'es are not supported. Fantastic. This wont stop me from building the array we desperately need here at home, but this sucks, in general.

What sort of issues were reported? Were people without systems administration skills building slow, congested networks with unmapped drives, causing NI samplers and apps to time out and lose mappings? Why is this (NAS vs local) not up to the user, including the performance liability if they should create it?

Kubrak

I guess NAS is not officially suported as one should not install NI SW on LAN. But I guess noone will kill you, if you install it on home NAS...

And people might face problems with weak NAS and slow connecions, which is not your case. And you have technical knowledge.....

If you have 10 Gbit LAN then it might be OK. I do not know what kind of sample libraries you use. Some Kontakt libraries are around 25 GB and one often uses more of them.... So even on 10 Gbit LAN loading data might take minute, two....

But it depends, what kind of music you do and what libraries you use. Anyway, no problem to install it on NAS and if it will not suite you, move it to computer.

Here comes how-to from an old NI forum:

https://www.native-instruments.com/forum/threads/ni-software-on-network-drives.321839/

Jeremy_NI

Hey @Milkman

Ok, so NAS'es are not supported. Fantastic. This wont stop me from building the array we desperately need here at home, but this sucks, in general.

NAS are not supported, it has nothing to do with licensing, it's just that NAS drives are not good with real time audio streams. NAS drives are still slower than physical ones and also more prone to errors. Native Access also doesn't support NAS drives. Issues reported are quite varied, from not being able to install the libraries or load them, audio drop outs, crackles, etc...You're free to do what you want with your Kontakt libraries, if issues come up, we will not be able to assist you. I hope you understand.

We made several articles on the subject:

Notes on Network Drives and Disk Formats

Native Access Error Message: "XML Processing Error"

Milkman

https://community.native-instruments.com/discussion/comment/4175#Comment_4175

I understand the i/o performance of NAS systems and networks, and if the only data that is stored across the network are sample libraries, what data would need to be realtime streamed to the sampler applications? Is 32GB system memory not adequate for RAM buffers of a size that would offset/hide this? I mean this would also be an issue for people with slow spinning disks connected locally, correct? I dont really care if my sample load times get worse -- dropping from less than 1 second, currently, to even 4-5 seconds. I am ok with that latency.

Thank you for the reply! I will check out those docs and probably try to make this work on my own, if possible. This is too important for my home network, and I simply cant keep buying new local HDDs for my main workstation. Already 7TB in there, and it is overutilized.

Im specc'ing out a large NAS as we speak, and this NAS is capable of transcoding 4k video across that same gigabit link, lol. It isnt even a NAS as much as a server at this point. ; ) Hope this all works.

Milkman

https://community.native-instruments.com/discussion/comment/4111#Comment_4111

All NI software, NA, Cubase, and a large array of plugins and tools are all installed locally, on fast SSDs. I just want to move the big stuff - groups/kits(samples) and all the small .nki and etc files to a fast, locally mapped NAS drive, leaving local program installs alone. The very low latency and good i/o performance I can measure across these links using iperf3(linux) suggest this should work, but would just slow down sample load times by a second or 2. I don't 100% understand why the application cannot be made to be tolerant of this, and/or to advise individual users of the potential liability they are inducing upon themselves. Im ok with the liability. =P

Milkman

https://community.native-instruments.com/discussion/comment/4175#Comment_4175

Oops, forgot to ask -- does NI have any future plans to support NAS on fast network links (specified by NI) for at least bulk sample libraries? Perhaps a new file system schema that is more tolerant of latency? (yeah, Im shooting for pie-in-the-sky here lol).

Kubrak

https://community.native-instruments.com/discussion/comment/4267#Comment_4267

I understand that you want to move big data to NAS. And if it is Expansions it will probably work fine, it is a lot data, but only small chunks need to be loaded.

32 GB of RAM may be OK, or may not be.... I had 8 GB, it was too low, 16 GB seemed to me enought. Soon I found, I need even more. So in PC that I built quite recently I have 64 GB. It is OK, so far, I use about 32 GB now, so there is headroom for even bigger projects....

Some of Komplete libraries are 25 GB, that may mean roughly 25 seconds download over 10 Gbit network. And one project usually has several of them....

I use five computers, and the one dedicated purely to music creation is by no means the strongest one, CPU and RAM wise. And it has triple the storage space (2x 1TB NVME SSD, 2x 2 TB HDD in RAID) than my very old NAS.....

Milkman

https://community.native-instruments.com/discussion/comment/4277#Comment_4277

Thanks for that. I havent gone over 32GB in a desktop environment on windows10 yet, so I wasnt entirely sure win10 pro would even properly manage 32GB+, but if you see windows10 (you on win10?) actually reserving this for paging, and you see applications taking advantage of this, then I might consider slapping in another 32GB, even just for testing. I wouldnt mind 64GB.

Even if a library is 25GB+ in total size, each patch, kit, or group wont be loading that full file size. It shouldnt be, anyway, imo. I use an Mdrummer library that is 150+GB in size, and when loading content just grabs the current data, and not the whole library.

At some point, I will be willing to upgrade to full 10Gbit on the network and all hosts. Cant quit manage that at this moment, but some day. Someone needs to make a fully switched eSata/network controller that routes all traffic over eSata at 600Gb/sec. And make it affordable, dangit! 😁 eSata router wooooo lmao

Kubrak

Yes, with kits and expansions no problem. They may be large, but many small files.

But I speak about Kontakt Libraries. There may consist of multi GB files for download.... But it aplies mostly to cinematic and symphonic libraries. Those are rather big... Would be nice to have them on NAS, but the loading times would be rather long even on 10 Gbit LAN...

If you do not use this kind of stuff, NAS would be most probably OK and 32 GB RAM also. Yes, I am on Win 10 with 64 GB RAM.

By the way, Win 10 Home handles up to 128 GB and Pro up to 2 TB RAM.... Even Win 7 Pro supports up to 192 GB. I remember times when 64 kB was overkill, but that was 40+ years ago...

Milkman

https://community.native-instruments.com/discussion/comment/4295#Comment_4295

I have avoided most symphonic libraries because 1) I dont typically need them, 2) they are some of the biggest libraries in terms of size.

Yeah, with RAM being cheaper overall these days, I'll probably go for 64 fairly soon, just for general principal if anything. Im on win10 pro and my MB can handle up 128, but I dont think I need 128 right now.

I'm going to give this a shot. I'm speccing out the NAS right now, and plan to buy sometime in March. I may buy the server itself, then buy the disks the month after so I can break up the cost. Ill come back and post my results once it is built.

Kubrak

Yes Symphonic libraries are huge, that is why I have two 1 TB NVMe SSDs... If you do not use Symphonic libraries or rarely, then NAS should be OK for your use case. Or you may install them on internal disc. But they are huge, e.g. Cremona Quartet almost 100 GB. So, putting them on NAS would be nice...

Someone has connected NAS using eSATA. That would be sort of acceptable compromise if one has really huge, huge libraries.

So, good luck.

Tocan

The argument that a NAS machine is to slow is total nonsense. In fact I run a NAS Server with a Double Xeon CPU means 60 cores, and all in all 1,5 TB of Ram. The network is done with a Broadcom Switch and have at least 40 GBe. Native Instruments seems to sleep now for 20 and more years. Because this is outdated snow of yesterday in enterprise it world why should this not run or be supported. Fact for production and security reasons there is no reason not to support network enterprise solutions. Part will be done on MACOS running Monterey at MAC OS and Parts on a Windows machine there is no reason for double download libaries twice.

I hope for an example how to make it possible to use a NAS System with Kontakt.

Quick links

Unanswered questions