mmap(MAP_FIXED) vs mprotect

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

mmap(MAP_FIXED) vs mprotect

Ryan Johnson-10
Hi all,

I'm trying to port a linux program that uses mmap to implement a
growable array; the ideas is to mmap(PROT_NONE, MAP_NORESERVE) a chunk
of address space (corresponding to the maximum array size) and then call
mmap(PROT_READ|PROT_WRITE, MAP_FIXED) to allocate actual memory in the
"blank" region. This works well in Linux but fails with EINVAL in cygwin.

My code aligns all sizes up to 2MB boundaries, so it's not a 64kB
boundary problem. My code reports the failing call as:
> 22 Invalid argument addr=0xffdb0000, sz=2097152

A peek in /proc/self/maps confirms that the address is correct:
> FFDB0000-FFFB0000 ===p 00000000 0000:0000 0

Oddly, trying to map in blank pages in with mprotect succeeds on cygwin
but fails with ENOMEM on linux...

Am I missing something here, or is this just a place where different
behavior between the two platforms is a fact of life? Which version is
the posixly "correct" way to reserve a chunk of address space and later
back it with actual memory?

Thanks,
Ryan


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Reply | Threaded
Open this post in threaded view
|

Re: mmap(MAP_FIXED) vs mprotect

Corinna Vinschen-2
On Apr 24 08:03, Ryan Johnson wrote:

> Hi all,
>
> I'm trying to port a linux program that uses mmap to implement a
> growable array; the ideas is to mmap(PROT_NONE, MAP_NORESERVE) a
> chunk of address space (corresponding to the maximum array size) and
> then call mmap(PROT_READ|PROT_WRITE, MAP_FIXED) to allocate actual
> memory in the "blank" region. This works well in Linux but fails
> with EINVAL in cygwin.
>
> My code aligns all sizes up to 2MB boundaries, so it's not a 64kB
> boundary problem. My code reports the failing call as:
> >22 Invalid argument addr=0xffdb0000, sz=2097152
>
> A peek in /proc/self/maps confirms that the address is correct:
> >FFDB0000-FFFB0000 ===p 00000000 0000:0000 0
>
> Oddly, trying to map in blank pages in with mprotect succeeds on
> cygwin but fails with ENOMEM on linux...
>
> Am I missing something here, or is this just a place where different
> behavior between the two platforms is a fact of life? Which version
> is the posixly "correct" way to reserve a chunk of address space and
> later back it with actual memory?

There is no POSIXly correct way to do that since MAP_NORESERVE is a
non-POSIX extension in Linux as well as in Cygwin.

The general idea of MAP_NORESERVE is to make sure that we get as much
memory as requested, but to use only as much memory as is required.
On Linux MAP_NORESERVE only performs bookkeeping but doesn't change
the state of the memory, so a later mmap works.  On Cygwin MAP_NORESERVE
uses the Windows way of handling this requirements, so in contrast to
what the name of the option suggests, Cygwin actually *reserves* space
but does not *commit* it.  Cygwin's mmap can't handle this, but you can
commit pages by using mprotect or by simply peeking or poking into
the address space.  This raises a SEGV, and the exception handling code
will then commit the page you peeked or poked.

Having said that, we *could* also change mmap to handle this scenario
gracefully as well.  http://cygwin.com/acronyms/#PTC.


Corinna

--
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Reply | Threaded
Open this post in threaded view
|

Re: mmap(MAP_FIXED) vs mprotect

Ryan Johnson-10
On 24/04/2013 8:50 AM, Corinna Vinschen wrote:
> On Apr 24 08:03, Ryan Johnson wrote:
>> Hi all,
>>
>> I'm trying to port a linux program that uses mmap to implement a
>> growable array; the ideas is to mmap(PROT_NONE, MAP_NORESERVE) a
>> chunk of address space (corresponding to the maximum array size) and
>> then call mmap(PROT_READ|PROT_WRITE, MAP_FIXED) to allocate actual
>> memory in the "blank" region. This works well in Linux but fails
>> with EINVAL in cygwin.
[snip]

>> Oddly, trying to map in blank pages in with mprotect succeeds on
>> cygwin but fails with ENOMEM on linux...
>>
>> Am I missing something here, or is this just a place where different
>> behavior between the two platforms is a fact of life? Which version
>> is the posixly "correct" way to reserve a chunk of address space and
>> later back it with actual memory?
> There is no POSIXly correct way to do that since MAP_NORESERVE is a
> non-POSIX extension in Linux as well as in Cygwin.
>
> The general idea of MAP_NORESERVE is to make sure that we get as much
> memory as requested, but to use only as much memory as is required.
> On Linux MAP_NORESERVE only performs bookkeeping but doesn't change
> the state of the memory, so a later mmap works.
Actually, MAP_NORESERVE is just to avoid getting OOM on machine with
strict accounting if multiple 16GB spaces get allocated in this way;
calling mmap with MAP_FIXED succeeds as long as the requested address
range has already been allocated to the process in some way. In fact,
the linux machine I tested on let me MAP_FIXED a 2MB region starting
from inside a 16kB mapping, with the unfortunate side effect of
clobbering the dynamic loader's state (which happened to be mapped
nearby).  That's a ridiculous amount of rope to hang yourself with, and
I'm not confident it's possible, let alone desirable, to implement it in
Cygwin (Windows refuses to unmap statically linked .dlls, for example).
Still, the ability to clobber anonymous memory regions with some other
mapping would be useful at times (e.g. to "concatenate" two files in
memory by mapping them side by side). To be safe, we could require that
region to be clobbered be PROT_NONE (which erases any data that may have
been there previously, thus guaranteeing that we don't lose anything
important by changing the mapping).
> Having said that, we *could* also change mmap to handle this scenario
> gracefully as well.  http://cygwin.com/acronyms/#PTC.
I guess hacking it in would be much less work than trying to improve
fork() was, but realistically I'm not going to have time to do it any
time soon (rats).

Ryan


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple