ulimit -n unlimited?
 

Hi,

I'm VERY new to OSX and the Terminal. I'm trying to go beyond the OSX open file limit of around 12000 for an application called Kontakt. I've heard various work arounds, but I just can't seem to implement them. Please be gentle with me as I'm almost a novice, but I have to nail this problem and quick.

Thanks in advance,

Andy.

p.s. Whenever I go into the Terminal and change a setting such as the 'ulimit -n' it will remember it until I start up the terminal again and will then default back to the original setting. I've tried logging on as the root user, but with the same result. I'm confused as to how to make it perminent.

Andy B,

Please, run in Terminal.app and post the result of:

ulimit -a -H


Does it help if you write in your ~/.bashrc 'ulimit -n 12000' ?

.

Hi Sao,

Thanks for the reply.

Here's the result of ulimit -a -H:
ulimit -a -H
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) unlimited
pipe size (512 bytes, -p) 1
stack size (kbytes, -s) 65536
cpu time (seconds, -t) unlimited
max user processes (-u) 532
virtual memory (kbytes, -v) 9007199254806527

Although it says open files unlimited, if I then run ulimit -n it says 256? I don't know what you mean by 'run in Terminal.app'. Is Terminal.app a window within the Terminal, or just the Terminal and if it is a window how do you create it?

I also don't understand if ~/.bashrc is a window or not.

All I know is that after running ulimit -a -H I still hit a open file limit within the kontakt application.

If you're pulling your hair out after reading this, I apologise!

Thanks,

Andy.

Is this "Kontakt" applications something that runs from the command line (inside a Terminal window) ?

Does Kontakt work after you do the 'ulimit -n 12000' ?
I.e. is your question only about how to make this change permanent?

To make it permanent, you could create a file named ".profile" (without the quotes) in your home folder (using a text editor - e.g. 'pico') and insert the following line into that file:
ulimit -n 12000

Note to others: ~/.bashrc is not read by default in login windows (as used by Terminal) - see this other post: http://forums.macosxhints.com/showpo...09&postcount=2




By the way:
Terminal.app is how some people refer to the Terminal application (esp. in order to distinguish it from other terminal emulator applications - e.g. xterm)

Hi Hayne,

OK seems like I've done my best to confuse everyone :o ! Kontakt is an audio sampling application, which streams audio samples from disk. In order to do this it has to load a small 24kb buffer of each sample into RAM. OSX obviously sees each buffer as an open file, so I can load about 10,500 of these buffers before the OS says no more because of the open file limit.
Simply, what I'm trying to achieve is lifting the limit so that I can have more samples loaded. At the moment when I reach the limit I'm only using about 300mb of RAM on a G5 Dual 2.5 with 4gigs of RAM - so you can see why I'm upset at hitting the open file limit :mad: .

I hope this makes things clearer now.

Thanks once again and I hope you'll be able to help.

Andy

By default, on my 768MB PowerBook at least, OS X allows a maximum of 10240 open files per process and a maximum of 12288 open files system wide. You can see these values by using the sysctl(8) command
You could try to increase these values until sufficient to run your file-hungry application (you need to be administrator)

I understood the gist of what you were trying to do before.
But you haven't answered my questions above.

Andy B,

Also, please run in Terminal.app and give us the following info:

sw_vers
echo $SHELL

Thanks again guys. Here's the result from Sao's last request:

sw_vers
ProductName: Mac OS X
ProductVersion: 10.3.5
BuildVersion: 7M34
G5-2-5:~ andrewblaney$ echo $SHELL
/bin/bash

"Is this "Kontakt" applications something that runs from the command line (inside a Terminal window) ?"

I don't completely understand the question, but I think the answer is that it isn't specific to the Terminal or you don't have to open the Terminal to run it. (hope that answers the question).

"Does Kontakt work after you do the 'ulimit -n 12000' ?"

Kontakt runs fine even if the ulimit -n is set to the default of 256. What's concerning is that no matter what I set the ulimit -n at, Kontakt always maxes out at around the just over 10,000 files open mark.

"I.e. is your question only about how to make this change permanent?"

That's part of it, but the ultimate question remains - How do I get past just over 10,000 files open (or even better - unlimited files open)?

Thanks,

Andy.

Okay, so now I think I understand that Kontakt is not a command-line application that you start from a Terminal window - it is a GUI application that you start by double-clicking on it. Is this correct?
If so, the ulimit settings that you configure in a shell (in Terminal) will have no effect on Kontakt.
All GUI applications are started via the WindowServer process and hence share its configuration settings. I'm not sure if there is a way to modify these settings aside from doing it in the system startup files, e.g. in /etc/rc
I wouldn't recommend changing these system files - at least not yet.

Instead, I think you should pursue a different tack. It doesn't seem reasonable to me for any application to have so many open files. So I recommend investigating more thoroughly the possibility of configuring Kontakt to avoid this. Check the developer's site & any available mailing lists, etc. And of course do some googling.

Thanks Hayne. Yes Kontakt is a GUI application (I didn't know this term).

I undersatnd you thinking that 10,000 open files within an application seems OTT, but there is simply no way around this. Each sample (file) creates part of a musical instrument preset. Within a preset you may have up to 30 articulations, within each articulation you may have up to 250 samples - so you can see how you can easily hit the 10,000 mark.

I've heard about the etc/rc command but always get a permission denied response, even when I log in as root (normally I'm Administrator). I've also heard that you can do a bit of damage messing around with it if you don't know what you're doing, but I'm getting quite desperate as this problem means that I'm only taking advantage of not even an eighth of my G5's current potential - so any help you could offer would be really appreciated.

Thanks,

Andy.

I can easily see how you might be using 10000 samples, but that doesn't mean that the application has to have 10000 files open. (It could open a file, get the contents, close the file - that is the usual way for files that are being read as opposed to written.)
I see that there is a user forum on the Sibelius web site, so you could ask your question there.

What you could try is to open a Terminal window, issue the appropriate 'ulimit' command to increase the allowable number of open files, and then launch the Kontakt application from the command line (in Terminal) by using the 'open -a' command.
E.g.:
open -a Kontakt

This makes me wonder: was my post earlier in this thread completely invisible, or just not clear enough? :)

It looks like hayne used Quote to reply to the message previous yours and everyone followed suit bypassing yours without the quote. BTW: I considered your reply right on the money. :)

I agree that 'sysctl' will likely provide a solution for Andy B's problem.

I had to ask 'Andy B' which system is he using, because 'kern.maxfilesperproc' is not changable in Jaguar. Apple implemented the 'kern.maxfilesperproc' in the Panther release. It seems it can only be edited in 10.3.x.

.

OK first of all apologies to Darelon. I skimmed your post as it looked like it was just confirming the max number of open files possible. I missed the important bit. However, I'm afraid I don't understand the advice. I ran what you suggested in the Terminal and here is the result:

sysctl(8)
-bash: syntax error near unexpected token `8'

It seems that most agree that this could be a fix for my problem so I'd be eager for some more pointers - once again sorry I missed it the first time.

"I can easily see how you might be using 10000 samples, but that doesn't mean that the application has to have 10000 files open. (It could open a file, get the contents, close the file - that is the usual way for files that are being read as opposed to written.) "

Thanks Hayne, but here's my understanding of why Kontakt can't do that (please correct me if I'm wrong!).

By opening the file - getting the contents - and then closing the file, I imagine it would have to store the contents in RAM. My problem is that I'm loading only the first 24kb of each file that acts as a pointer to the rest of the file, so that when I trigger that file from my MIDI keyboard the 24kb buffer addresses the rest of the file which it streams from disk. In order for Kontakt to be able to close the file I think it would have to load the complete contents into RAM and this would defeat the whole point of being able to stream from disk (because I would run out of RAM very quickly).

There is another possible answer to my problem that I've just discovered and it's that Kontakt may have an open file limit built into it using the setrlimit. Is this feasible and how easy would it be to adjust?

Finally, I've tried posting the problem on numerous music software sites (including Kontakt's), but at the end of the day the people on those sites are like me - they know enough about their Macs to use them to write music, but for my problem it needs people who really know what they're talking about - which is why I'm posting here.

Thanks again for your time and patience.

Andy.

Just tried this and unfortunately no go. Does this point towards it being an application limit?

Andy.

The commands that darelon was suggesting you run are the text inside the "code" boxes - but without the initial $
The first command:

sysctl kern.maxfiles kern.maxfilesperproc

will show you the current system limits.

The second command (which will prompt you for your admin password):

sudo sysctl -w kern.maxfilesperproc=20480 kern.maxfiles=22528

will change the limits to the numbers shown.
maxfiles is the maximum total number of open files allowed on the system as a whole
maxfilesperproc is the maximum number of open files allowed for any one program

Andy B,

Also, Panther supports using /etc/sysctl.conf to set sysctl values. The /etc/rc startup script is setup to read from the /etc/sysctl.conf file which it doesn't exist by default. You could create a 'sysctl.conf' file and put your changes there by running in Terminal.app:

% sudo pico /etc/sysctl.conf

and then writing the following lines to it:

kern.maxfilesperproc=20480
kern.maxfiles=22528


You can also check the current system limits with:

.

Just tried Hayne and Sao's latest suggestions. Everything worked in the Terminal, but unfortunately it made no difference to how much I could load into Kontakt. Surely this now suggests that the open file limit is written into the Kontakt software? If this is the case, could this help:
http://developer.apple.com/qa/qa2001/qa1005.html
or if not does anyone have any ideas?

Thanks again,

Andy.

I repeat my advice that you should ask the developer and/or the forums at the developer's site. There may be another way to accomplish what you are trying to do with the software that doesn't involve the huge number of open files. I.e. try to solve it by using the Kontakt application differently.

OK, fair enough. Thanks for all of the time and energy you've put into this.

Andy.

Fix for tcsh
 

Hi, folks...

I was running into a similar problem with simple scripts, running in tcsh. For whatever reason, the open file limits for tcsh were set to 256...quite a low number. Using the "limit descriptors unlimited" command (can also be placed in your .cshrc file), the number went from 256 to 10240. Getting beyond 10240, as discussed at length above, is a hairier issue. If you just need to hit the 4- or 5-digit regime, however, this is an easy fix.

So late
 

Hi there,

I am sorry that I am joining that late, but I have Kontakt, too... ;-)

So, at the moment I am really getting confused. I am using Logic with that Kontakt as an audio plugin. That error -42 comes up to fast, the cpu is hardly at 1/3 and the ram has also enough space (as andy b described above). So, I can't use my fast computer because of a software programming thing?! Has there anything happened about that topic? Is there a way around?

I tried that:
sudo sysctl -w kern.maxfiles=80000
sudo sysctl -w kern.maxfilesperproc=60000

... in the end but even that isn't enough. I have opened about 12000 Files within Logic, concerning EXS Samples and these Kontakt stuff, so it seems to have no effect at all, but the terminal says it is set to 80000 and 60000. I just need way less, but I thought who knows, perhaps that could do the trick.

OK, hard day for me, long post though, sorry! Is there any way to open that limit? I am running OSX.4.9

Regards,
Xcell

I can change these settings fine on my system. My only question is do you think this will help speed up Final Cut Pro? How can I tell how many files the application is using... I have noticed that even when rendering huge files with final cut being the only application open it still doesn't max out my 8 gig quad core... Is it possible to make it use all memory that isn't tied up by the OS...