/dev/fd/N not synonymous with file descriptor N; it is on Linux

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

/dev/fd/N not synonymous with file descriptor N; it is on Linux

Houder
L.S.,

/dev/fd/N not synonymous with file descriptor N; it is on Linux

64-@@ cat /dev/fd/0 <<\EOF
> Hi
> EOF
cat: /dev/fd/0: No such file or directory

fails on Cygwin; not on Linux.

Also see:

     https://cygwin.com/ml/cygwin/2018-12/msg00028.html
     ( Bash heredoc on FD 3 )

Based on the output of strace on Linux, I composed an STC, that
duplicates
the steps taken by bash (and cat).

This STC succeeds on Linux, but fails on Cygwin.

What does the STC do:

  - it creates a (temporary) file in the same way that bash does
  - the file is written to, the file descriptor is closed and the file
unlinked
  - however, before the file is unlinked, it is opened a second time,
like bash
    would do

Next
  - the file /dev/fd/N is opened, where N is the file descriptor that has
been
    left open; this is what the "cat command" would do.

The "cat command" on Linux succeeds; it fails on Cygwin.

Regards,
Henri

STC attached (hopefully)

=====

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Corinna Vinschen-2
On Dec 16 17:31, Houder wrote:
> L.S.,
>
> /dev/fd/N not synonymous with file descriptor N; it is on Linux

Yes, it is.  Most of the time.  Try this:

$ echo foo | cat /dev/fd/0

The problem is that some of the concepts don't work as desired:

> 64-@@ cat /dev/fd/0 <<\EOF

If you observe what happens in tcsh in this situation you see that it
doesn't even execute cat as long as you didn't type EOF.  What you type
is written to a tmpfile:

$ ls -l /proc/5980/fd
total 0
lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 0 -> /tmp/sh.lVQq04
lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 15 -> /dev/pty0
lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 16 -> /dev/pty0
lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 17 -> /dev/pty0
lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 18 -> /dev/pty0
lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 19 -> /dev/pty0

However, this tmpfile has been unlinked already, so it has been moved to the
recycle bin:

$ ls -l /tmp/sh.lVQq04
ls: /tmp/sh.lVQq04: No such file or directory

So the path in the fd subdir doesn't reflect the actual file path.

But after starting cat, cat tries to open /proc/self/fd/0 which
is in fact the non-existing path /tmp/sh.lVQq04.  Bad luck.

In contrast to Linux the symlinks are not just faked symlinks with the
underlying OS having direct access to the file descriptors.  The way
it's implemented in Cygwin uses the actual file path resolution and then
either works or fails as above.  I'm not sure how to fix that easily.
I guess the fd/0 symlink would have to show the actual file path pointing
to the recycle bin.  But that's often not what you want either since it
hows a patch outside the POSIX namespace.


Corinna

--
Corinna Vinschen
Cygwin Maintainer

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Corinna Vinschen-2
On Dec 16 21:28, Corinna Vinschen wrote:

> On Dec 16 17:31, Houder wrote:
> > L.S.,
> >
> > /dev/fd/N not synonymous with file descriptor N; it is on Linux
>
> Yes, it is.  Most of the time.  Try this:
>
> $ echo foo | cat /dev/fd/0
>
> The problem is that some of the concepts don't work as desired:
>
> > 64-@@ cat /dev/fd/0 <<\EOF
>
> If you observe what happens in tcsh in this situation you see that it
> doesn't even execute cat as long as you didn't type EOF.  What you type
> is written to a tmpfile:
>
> $ ls -l /proc/5980/fd
> total 0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 0 -> /tmp/sh.lVQq04
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 15 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 16 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 17 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 18 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 19 -> /dev/pty0
>
> However, this tmpfile has been unlinked already, so it has been moved to the
> recycle bin:
>
> $ ls -l /tmp/sh.lVQq04
> ls: /tmp/sh.lVQq04: No such file or directory
>
> So the path in the fd subdir doesn't reflect the actual file path.
>
> But after starting cat, cat tries to open /proc/self/fd/0 which
> is in fact the non-existing path /tmp/sh.lVQq04.  Bad luck.
>
> In contrast to Linux the symlinks are not just faked symlinks with the
> underlying OS having direct access to the file descriptors.  The way
> it's implemented in Cygwin uses the actual file path resolution and then
> either works or fails as above.  I'm not sure how to fix that easily.
> I guess the fd/0 symlink would have to show the actual file path pointing
> to the recycle bin.  But that's often not what you want either since it
> hows a patch outside the POSIX namespace.
  shows a path

Sorry,
Corinna

--
Corinna Vinschen
Cygwin Maintainer

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Wayne Davison
In reply to this post by Corinna Vinschen-2
On Sun, Dec 16, 2018 at 12:29 PM Corinna Vinschen wrote:
> In contrast to Linux the symlinks are not just faked symlinks with the underlying OS having direct access to the file descriptors.

Yeah, Linux is more like a fuse where the open filehandles are used
directly on open, and the stat calls return pretend symlinks with the
lsof info.

> The way it's implemented in Cygwin uses the actual file path resolution and then either works or fails as above.

If Cygwin can't be changed into a fuse idiom, perhaps a hard-link
idiom could be used? For instance, when the /proc/$PID/fd/0 symlink is
being created, the code could try to hard-link the file to
/proc/$PID/.fd/0 first (note the dot-fd) before creating the normal
fd/0 symlink. Then, when a request came in to read /proc/$PID/fd/0, it
could check if the associated hard-link exists and read from that
instead.  Or perhaps the "symlink" info could be stored elsewhere and
the fd/0 file could be the hard-link, requiring the stat code to be
tweaked to look up the pretend-symlink info?

..wayne..

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Corinna Vinschen-2
On Dec 16 13:36, Wayne Davison wrote:

> On Sun, Dec 16, 2018 at 12:29 PM Corinna Vinschen wrote:
> > In contrast to Linux the symlinks are not just faked symlinks with the underlying OS having direct access to the file descriptors.
>
> Yeah, Linux is more like a fuse where the open filehandles are used
> directly on open, and the stat calls return pretend symlinks with the
> lsof info.
>
> > The way it's implemented in Cygwin uses the actual file path resolution and then either works or fails as above.
>
> If Cygwin can't be changed into a fuse idiom, perhaps a hard-link
> idiom could be used? For instance, when the /proc/$PID/fd/0 symlink is
> being created, the code could try to hard-link the file to
> /proc/$PID/.fd/0 first
Nice try, but hardlinks don't cross FS borders.  /proc is a filesystem
on its own with its own inode numbers.

I'm mulling over adding some hack to open().  It could try to recognize
the special case of opening a processes' own descriptor symlink within
/proc and then warp the open() call into dup().  No idea how tricky
or even feasible that is, though...


Corinna

--
Corinna Vinschen
Cygwin Maintainer

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Houder
In reply to this post by Corinna Vinschen-2
On 2018-12-16 21:28, Corinna Vinschen wrote:

> On Dec 16 17:31, Houder wrote:
>> L.S.,
>>
>> /dev/fd/N not synonymous with file descriptor N; it is on Linux
>
> Yes, it is.  Most of the time.  Try this:
>
> $ echo foo | cat /dev/fd/0
>
> The problem is that some of the concepts don't work as desired:
>
>> 64-@@ cat /dev/fd/0 <<\EOF
>
> If you observe what happens in tcsh in this situation you see that it
> doesn't even execute cat as long as you didn't type EOF.  What you type
> is written to a tmpfile:
>
> $ ls -l /proc/5980/fd
> total 0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 0 -> /tmp/sh.lVQq04
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 15 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 16 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 17 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 18 -> /dev/pty0
> lrwxrwxrwx 1 corinna vinschen 0 Dec 16 21:15 19 -> /dev/pty0
>
> However, this tmpfile has been unlinked already, so it has been moved
> to the
> recycle bin:
>
> $ ls -l /tmp/sh.lVQq04
> ls: /tmp/sh.lVQq04: No such file or directory
>
> So the path in the fd subdir doesn't reflect the actual file path.
>
> But after starting cat, cat tries to open /proc/self/fd/0 which
> is in fact the non-existing path /tmp/sh.lVQq04.  Bad luck.
Yes Corinna, I am aware of the above. I described it here:

     https://cygwin.com/ml/cygwin/2018-12/msg00040.html
     ( Re: Bash heredoc on FD 3 )

Sorry for NOT expressing myself more clearly.

The STC is, as I wrote, based on what I observed on Linux.

See attachment.

Regards,
Henri

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

disassembly.txt (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Houder
In reply to this post by Corinna Vinschen-2
On 2018-12-16 22:55, Corinna Vinschen wrote:
[snip]

> I'm mulling over adding some hack to open().  It could try to recognize
> the special case of opening a processes' own descriptor symlink within
> /proc and then warp the open() call into dup().  No idea how tricky
> or even feasible that is, though...

That is why I wrote the following in my STC:

         // Q: does Cygwin attempt to read the /tmp directory? (an
attempt that
         //    will fail, because the file has been unlinked)
         // it appears that reading a symlnk in /dev/fd can best be
diverted to
         // the open file descriptor of the process ...

What I meant was, that I see no reason to modify the symlink in this
special case, but in stead of that to access the file using fd N, where
N is equal to the one in /dev/fd/N.

File descriptor N has been left open by bash and should not have been
closed as result of the exec ...

And indeed, I have _no_ clue if the above is feasible (and tricky?) in
Cygwin; otherwise I would have posted a solution.

Regards,
Henri

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Corinna Vinschen-2
On Dec 17 05:30, Houder wrote:

> On 2018-12-16 22:55, Corinna Vinschen wrote:
> [snip]
>
> > I'm mulling over adding some hack to open().  It could try to recognize
> > the special case of opening a processes' own descriptor symlink within
> > /proc and then warp the open() call into dup().  No idea how tricky
> > or even feasible that is, though...
>
> That is why I wrote the following in my STC:
>
>         // Q: does Cygwin attempt to read the /tmp directory? (an attempt
> that
>         //    will fail, because the file has been unlinked)
>         // it appears that reading a symlnk in /dev/fd can best be diverted
> to
>         // the open file descriptor of the process ...
>
> What I meant was, that I see no reason to modify the symlink in this
> special case, but in stead of that to access the file using fd N, where
> N is equal to the one in /dev/fd/N.
>
> File descriptor N has been left open by bash and should not have been
> closed as result of the exec ...
tl;dr:

  cat tries to open /dev/fd/0.  /dev/fd/0 refers to a non-existing
  file and that's why open fails.  Cygwin doesn't have special code to
  handle the fd symlinks as refference to an actually open descriptors.

The long story:

  The descriptor hasn't been closed.  Cat still has the file open as fd 0.

  The problem is that you tell cat to open and read from /dev/fd/0,
  *not* from fd 0.  That's quite a difference.  /dev/fd/0 is just some
  arbitrary file which is given to open(2), a symlink at that.  First
  thing open(2) does is to call the symlink evaluation code.  The
  symlink is ultimately evaluated to the filename opened by the shell.

  So, cat's open(2) tries to open "/tmp/whatever".  But, as you noticed,
  that path doesn't exist anymore since it has been deleted by the shell
  before even writing to it via unlink(2).

  What happens is that unlink(2) is supposed to delete a file in use.
  Since that's not possible in Windows, the file gets renamed into the
  recycle bin and just marked for deletion.

  The fact that the filename changed is not known to the application of
  course, it just gets success reported from unlink.  Cygwin itself
  doesn't know the new name of the file either.  There just isn't a
  reason to keep track since it's going to be removed anyway at one
  point.

  And even *if* we keep track, we can't use this information since files
  marked for deletion can't be opened on Windows (ERROR_DELETE_PENDING
  or ERROR_ACCESS_DENIED, I'm not sure which).

  Here's another problem with handling /dev/fd/0 as just some descriptor
  we can call dup(2) on:

  /dev/fd/0 is just some symlink which evaluates like this:

  - /dev/fd is a symlink to /proc/self/fd
  - /proc/self is a symlink to /proc/<current_process_pid>

  So before we even get to evaluate fd 0 symlink, we're at

    /proc/<current_process_pid>/fd/0

  Here's the question:  What if you don't give /proc/self/fd/0 to
  cat, but something like /proc/<some_other_processes_pid>/fd/0?

  That's where Cygwin just fails because /proc is a process-local fake
  in our user space Cygwin DLL.  Emulating /proc is fun stuff, the full
  functionality is pretty tricky without going to great lengths like
  starting a service with SYSTEM permissions.

  Bottom line is, we could do more if we decide that running cygserver
  is a requirement.  As it is, we always strived for a setup which
  works "out of the box", without having to start services and stuff.


Corinna

--
Corinna Vinschen
Cygwin Maintainer

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Corinna Vinschen-2
On Dec 17 10:25, Corinna Vinschen wrote:

> On Dec 17 05:30, Houder wrote:
> > On 2018-12-16 22:55, Corinna Vinschen wrote:
> > [snip]
> >
> > > I'm mulling over adding some hack to open().  It could try to recognize
> > > the special case of opening a processes' own descriptor symlink within
> > > /proc and then warp the open() call into dup().  No idea how tricky
> > > or even feasible that is, though...
> >
> > That is why I wrote the following in my STC:
> >
> >         // Q: does Cygwin attempt to read the /tmp directory? (an attempt
> > that
> >         //    will fail, because the file has been unlinked)
> >         // it appears that reading a symlnk in /dev/fd can best be diverted
> > to
> >         // the open file descriptor of the process ...
> >
> > What I meant was, that I see no reason to modify the symlink in this
> > special case, but in stead of that to access the file using fd N, where
> > N is equal to the one in /dev/fd/N.
> >
> > File descriptor N has been left open by bash and should not have been
> > closed as result of the exec ...
>
> tl;dr:
>
>   cat tries to open /dev/fd/0.  /dev/fd/0 refers to a non-existing
>   file and that's why open fails.  Cygwin doesn't have special code to
>   handle the fd symlinks as refference to an actually open descriptors.
>
> The long story:
>
>   The descriptor hasn't been closed.  Cat still has the file open as fd 0.
>
>   The problem is that you tell cat to open and read from /dev/fd/0,
>   *not* from fd 0.  That's quite a difference.  /dev/fd/0 is just some
>   arbitrary file which is given to open(2), a symlink at that.  First
>   thing open(2) does is to call the symlink evaluation code.  The
>   symlink is ultimately evaluated to the filename opened by the shell.
I just tested this on Linux.  Basically, open(2) in Cygwin is equivalent
to calling readlink(2); open(2):

  readlink(incoming_filename, resolved_filename, ...);
  open (resolved_filename, ...);

If I do this on Linux with a cat-clone calling readlink before calling
open, the error returned by open(2) is the same as on Cygwin, ENOENT.

The tricky part is to find out when resolving the filename is not
required because we already have a descriptor / HANDLE and how to
proceed from there...


Corinna

--
Corinna Vinschen
Cygwin Maintainer

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Houder
On 2018-12-17 12:26, Corinna Vinschen wrote:

> On Dec 17 10:25, Corinna Vinschen wrote:
>> On Dec 17 05:30, Houder wrote:
>> > On 2018-12-16 22:55, Corinna Vinschen wrote:
>> > [snip]
>> >
>> > > I'm mulling over adding some hack to open().  It could try to recognize
>> > > the special case of opening a processes' own descriptor symlink within
>> > > /proc and then warp the open() call into dup().  No idea how tricky
>> > > or even feasible that is, though...
>> >
>> > That is why I wrote the following in my STC:
>> >
>> >         // Q: does Cygwin attempt to read the /tmp directory? (an attempt
>> > that
>> >         //    will fail, because the file has been unlinked)
>> >         // it appears that reading a symlnk in /dev/fd can best be diverted
>> > to
>> >         // the open file descriptor of the process ...
>> >
>> > What I meant was, that I see no reason to modify the symlink in this
>> > special case, but in stead of that to access the file using fd N, where
>> > N is equal to the one in /dev/fd/N.
>> >
>> > File descriptor N has been left open by bash and should not have been
>> > closed as result of the exec ...
[snip]

> The tricky part is to find out when resolving the filename is not
> required because we already have a descriptor / HANDLE and how to
> proceed from there...

Ref: https://cygwin.com/ml/cygwin/2018-12/msg00125.html
( /dev/fd/N not synonymous with file descriptor N; it is on Linux )

For the record only:

You were/are right and I was complete wrong: open file descriptor N is
only by chance equal to /dev/fd/N ...

Neither the semantics of FreeBSD (and Solaris) -- similar to dup() --
nor those of Linux (where the opening of the deleted file results in a
new entry in the open file table) are feasible in Cygwin.

FreeBSD (Solaris) and Linux require the concept of a "directory cache"
to exist in order to make the above possible (that cache must also be
exposed to the user in some way).

FreeBSD (Solaris): based on the filename, the corresponding open file
descriptION is found; a new file descriptor to this entry in the open
file table is returned to the user ...

It appears as if the inherited open file descriptor is dup'ed, but it
is not.

Linux: based on the filename, the corresponding inode is found; a new
entry in the open file table is created; a file descriptor to the new
entry in the open file table is returned to the user ...
(this can be verified using kcmp(2) ).

The above is not possible in Cygwin. Cygwin cannot follow here.

Regards,
Henri

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Corinna Vinschen-2
In reply to this post by Houder
On Dec 16 17:31, Houder wrote:

> L.S.,
>
> /dev/fd/N not synonymous with file descriptor N; it is on Linux
>
> 64-@@ cat /dev/fd/0 <<\EOF
> > Hi
> > EOF
> cat: /dev/fd/0: No such file or directory
>
> fails on Cygwin; not on Linux.
>
> Also see:
>
>     https://cygwin.com/ml/cygwin/2018-12/msg00028.html
>     ( Bash heredoc on FD 3 )
>
> Based on the output of strace on Linux, I composed an STC, that duplicates
> the steps taken by bash (and cat).
This should work in the latest developer snapshot uploaded to
https://cygwin.com/snapshots/  Please give it a try.


Thanks,
Corinna

--
Corinna Vinschen
Cygwin Maintainer

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Houder
On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:

>
> On Dec 16 17:31, Houder wrote:
> > L.S.,
> >
> > /dev/fd/N not synonymous with file descriptor N; it is on Linux
> >
> > 64-@@ cat /dev/fd/0 <<\EOF
> > > Hi
> > > EOF
> > cat: /dev/fd/0: No such file or directory
> >
> > fails on Cygwin; not on Linux.
> >
> > Also see:
> >
> >     https://cygwin.com/ml/cygwin/2018-12/msg00028.html
> >     ( Bash heredoc on FD 3 )
> >
> > Based on the output of strace on Linux, I composed an STC, that duplicates
> > the steps taken by bash (and cat).
>
> This should work in the latest developer snapshot uploaded to
> https://cygwin.com/snapshots/  Please give it a try.
>
> Thanks,
> Corinna

Nice!

This solves: <program> -i /dev/fd/N N<<EOF (i.e. Steven Penny's problem)

Howver ...
(and I sure that I am not telling you anything new)

it still not the same as Linux ...

So, for the record only:

 - suppose a hacker deletes the logfile, but the file is still "help open"
   by some program
 - the logfile can be rescued on Linux, but not on Cygwin

64-@@ echo aap > aap.txt
64-@@ tail -f aap.txt
aap

.. in another terminal
64-@@ ls -l /proc/4120/fd # 4120 is the procid of "tail"
total 0
lrwxrwxrwx 1 Henri None 0 Jan 22 09:25 0 -> /dev/pty1
lrwxrwxrwx 1 Henri None 0 Jan 22 09:25 1 -> /dev/pty1
lrwxrwxrwx 1 Henri None 0 Jan 22 09:25 2 -> /dev/pty1
lrwxrwxrwx 1 Henri None 0 Jan 22 09:25 3 -> /home/Henri/redirect/aap.txt

64-@@ stat -L /proc/4120/fd/3 # shows a file of 4 bytes ...
  File: /proc/4120/fd/3
  Size: 4               Blocks: 1          IO Block: 65536  regular file
Device: 33d91880h/869865600d    Inode: 35747322042382612  Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/   Henri)   Gid: (  513/    None)
Access: 2019-01-22 09:24:41.978178200 +0100
Modify: 2019-01-22 09:24:41.978178200 +0100
Change: 2019-01-22 09:24:41.978178200 +0100
 Birth: 2019-01-22 09:24:41.978178200 +0100

64-@@ rm aap.txt # hacker removing the logfile

64-@@ stat -L /proc/4120/fd/3
  File: /proc/4120/fd/3
  Size: 4               Blocks: 1          IO Block: 65536  regular file
Device: 33d91880h/869865600d    Inode: 35747322042382612  Links: 0
Access: (0644/-rw-r--r--)  Uid: ( 1000/   Henri)   Gid: (  513/    None)
Access: 2019-01-22 09:24:41.978178200 +0100
Modify: 2019-01-22 09:24:41.978178200 +0100
Change: 2019-01-22 09:26:25.021040800 +0100
 Birth: 2019-01-22 09:24:41.978178200 +0100

64-@@ cp /proc/4120/fd/3 noot.txt # logfile rescue
64-@@ cat noot.txt # empty! (however, this works on Linux)
64-@@ cat /proc/4120/fd/3 # indeed, does not show contents
64-@@

=====


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Houder
On 2019-01-22 09:50, Houder wrote:

> On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:
>>
>> On Dec 16 17:31, Houder wrote:
>> > L.S.,
>> >
>> > /dev/fd/N not synonymous with file descriptor N; it is on Linux
>> >
>> > 64-@@ cat /dev/fd/0 <<\EOF
>> > > Hi
>> > > EOF
>> > cat: /dev/fd/0: No such file or directory
>> >
>> > fails on Cygwin; not on Linux.
>> >
>> > Also see:
>> >
>> >     https://cygwin.com/ml/cygwin/2018-12/msg00028.html
>> >     ( Bash heredoc on FD 3 )
>> >
>> > Based on the output of strace on Linux, I composed an STC, that duplicates
>> > the steps taken by bash (and cat).
>>
>> This should work in the latest developer snapshot uploaded to
>> https://cygwin.com/snapshots/  Please give it a try.
>>
>> Thanks,
>> Corinna
>
> Nice!
>
> This solves: <program> -i /dev/fd/N N<<EOF (i.e. Steven Penny's
> problem)
>
> Howver ...
> (and I sure that I am not telling you anything new)
>
> it still not the same as Linux ...
>
> So, for the record only:
and as another example, this STC succeeds on Linux ..., but fails on
Cygwin.

64-@@ ./stca /dev/fd/0 <<EOF
> bla
> EOF
fd1 = 0
argv[1] = /dev/fd/0
fd2 = 3
id = writefd2, errno = 13, errstr = Permission denied
64-@@

=====



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

stca.c (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Corinna Vinschen-2
In reply to this post by Houder
On Jan 22 09:50, Houder wrote:

> On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:
> > > Also see:
> > >
> > >     https://cygwin.com/ml/cygwin/2018-12/msg00028.html
> > >     ( Bash heredoc on FD 3 )
> > >
> > > Based on the output of strace on Linux, I composed an STC, that duplicates
> > > the steps taken by bash (and cat).
> >
> > This should work in the latest developer snapshot uploaded to
> > https://cygwin.com/snapshots/  Please give it a try.
>
> This solves: <program> -i /dev/fd/N N<<EOF (i.e. Steven Penny's problem)
>
> Howver ...
> (and I sure that I am not telling you anything new)
> [...]
> 64-@@ rm aap.txt # hacker removing the logfile
>
> 64-@@ stat -L /proc/4120/fd/3
>   File: /proc/4120/fd/3
>   Size: 4               Blocks: 1          IO Block: 65536  regular file
> Device: 33d91880h/869865600d    Inode: 35747322042382612  Links: 0
> Access: (0644/-rw-r--r--)  Uid: ( 1000/   Henri)   Gid: (  513/    None)
> Access: 2019-01-22 09:24:41.978178200 +0100
> Modify: 2019-01-22 09:24:41.978178200 +0100
> Change: 2019-01-22 09:26:25.021040800 +0100
>  Birth: 2019-01-22 09:24:41.978178200 +0100
>
> 64-@@ cp /proc/4120/fd/3 noot.txt # logfile rescue
> 64-@@ cat noot.txt # empty! (however, this works on Linux)
> 64-@@ cat /proc/4120/fd/3 # indeed, does not show contents
> 64-@@
Yes, this does not work under older systems.  However, it works as
desired at least with Windows 10 1709 due to changes in the OS in
terms of handling of deleted files.

I tested this with W8.1, W10 1709 and W10 1809.  It fails on 8.1
but works fine on both W10 versions.


Corinna

--
Corinna Vinschen
Cygwin Maintainer

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Corinna Vinschen-2
In reply to this post by Houder
On Jan 22 09:57, Houder wrote:

> On 2019-01-22 09:50, Houder wrote:
> > On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:
> > > This should work in the latest developer snapshot uploaded to
> > > https://cygwin.com/snapshots/  Please give it a try.
> > So, for the record only:
>
> and as another example, this STC succeeds on Linux ..., but fails on Cygwin.
>
> 64-@@ ./stca /dev/fd/0 <<EOF
> > bla
> > EOF
> fd1 = 0
> argv[1] = /dev/fd/0
> fd2 = 3
> id = writefd2, errno = 13, errstr = Permission denied
> 64-@@
Not sure what you're testing.  This is the result for me on both,
Windows 8.1 and Windows 10 1809:

)$ ./stca /dev/fd/0 <<EOF
? bla
? EOF
fd1 = 0
argv[1] = /dev/fd/0
fd2 = 3
buf = \
Hello, world!


Corinna

--
Corinna Vinschen
Cygwin Maintainer

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Corinna Vinschen-2
In reply to this post by Corinna Vinschen-2
On Jan 22 10:02, Corinna Vinschen wrote:

> On Jan 22 09:50, Houder wrote:
> > On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:
> > > > Also see:
> > > >
> > > >     https://cygwin.com/ml/cygwin/2018-12/msg00028.html
> > > >     ( Bash heredoc on FD 3 )
> > > >
> > > > Based on the output of strace on Linux, I composed an STC, that duplicates
> > > > the steps taken by bash (and cat).
> > >
> > > This should work in the latest developer snapshot uploaded to
> > > https://cygwin.com/snapshots/  Please give it a try.
> >
> > This solves: <program> -i /dev/fd/N N<<EOF (i.e. Steven Penny's problem)
> >
> > Howver ...
> > (and I sure that I am not telling you anything new)
> > [...]
> > 64-@@ rm aap.txt # hacker removing the logfile
> >
> > 64-@@ stat -L /proc/4120/fd/3
> >   File: /proc/4120/fd/3
> >   Size: 4               Blocks: 1          IO Block: 65536  regular file
> > Device: 33d91880h/869865600d    Inode: 35747322042382612  Links: 0
> > Access: (0644/-rw-r--r--)  Uid: ( 1000/   Henri)   Gid: (  513/    None)
> > Access: 2019-01-22 09:24:41.978178200 +0100
> > Modify: 2019-01-22 09:24:41.978178200 +0100
> > Change: 2019-01-22 09:26:25.021040800 +0100
> >  Birth: 2019-01-22 09:24:41.978178200 +0100
> >
> > 64-@@ cp /proc/4120/fd/3 noot.txt # logfile rescue
> > 64-@@ cat noot.txt # empty! (however, this works on Linux)
> > 64-@@ cat /proc/4120/fd/3 # indeed, does not show contents
> > 64-@@
>
> Yes, this does not work under older systems.  However, it works as
> desired at least with Windows 10 1709 due to changes in the OS in
> terms of handling of deleted files.
>
> I tested this with W8.1, W10 1709 and W10 1809.  It fails on 8.1
... as expected ...

> but works fine on both W10 versions.

Corinna

--
Corinna Vinschen
Cygwin Maintainer

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Houder
In reply to this post by Corinna Vinschen-2
On 2019-01-22 10:06, Corinna Vinschen wrote:

> On Jan 22 09:57, Houder wrote:
>> On 2019-01-22 09:50, Houder wrote:
>> > On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:
>> > > This should work in the latest developer snapshot uploaded to
>> > > https://cygwin.com/snapshots/  Please give it a try.
>> > So, for the record only:
>>
>> and as another example, this STC succeeds on Linux ..., but fails on
>> Cygwin.
>>
>> 64-@@ ./stca /dev/fd/0 <<EOF
>> > bla
>> > EOF
>> fd1 = 0
>> argv[1] = /dev/fd/0
>> fd2 = 3
>> id = writefd2, errno = 13, errstr = Permission denied
>> 64-@@
>
> Not sure what you're testing.  This is the result for me on both,
> Windows 8.1 and Windows 10 1809:

Curious! It fails (for me) on W7 ...

> Not sure what you're testing.

STC inherits a "read-only" open file descriptor from bash. On Linux
the file can be opened read-write (via procfs), because a new entry
is created in the open file table.

(opening the file read-write (via fdescfs) on FreeBSD would fail)

For this reason the output does not show what has been entered via
the here-doc.

In short, I was merely testing the semantics of Linux.

> )$ ./stca /dev/fd/0 <<EOF
> ? bla
> ? EOF
> fd1 = 0
> argv[1] = /dev/fd/0
> fd2 = 3
> buf = \
> Hello, world!
>
>
> Corinna

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Corinna Vinschen-2
On Jan 22 10:25, Houder wrote:

> On 2019-01-22 10:06, Corinna Vinschen wrote:
> > On Jan 22 09:57, Houder wrote:
> > > On 2019-01-22 09:50, Houder wrote:
> > > > On Sun, 6 Jan 2019 21:19:50, Corinna Vinschen  wrote:
> > > > > This should work in the latest developer snapshot uploaded to
> > > > > https://cygwin.com/snapshots/  Please give it a try.
> > > > So, for the record only:
> > >
> > > and as another example, this STC succeeds on Linux ..., but fails on
> > > Cygwin.
> > >
> > > 64-@@ ./stca /dev/fd/0 <<EOF
> > > > bla
> > > > EOF
> > > fd1 = 0
> > > argv[1] = /dev/fd/0
> > > fd2 = 3
> > > id = writefd2, errno = 13, errstr = Permission denied
> > > 64-@@
> >
> > Not sure what you're testing.  This is the result for me on both,
> > Windows 8.1 and Windows 10 1809:
>
> Curious! It fails (for me) on W7 ...
It works for me just as well on W7:

$ uname -a
CYGWIN_NT-6.1 vmbert764 2.12.0(0.333/5/3) 2019-01-21 22:47 x86_64 Cygwin
$ ./stca /dev/fd/0 <<EOF
? bla
? EOF
fd1 = 0
argv[1] = /dev/fd/0
fd2 = 3
buf = \
Hello, world!

> > Not sure what you're testing.
>
> STC inherits a "read-only" open file descriptor from bash. On Linux
> the file can be opened read-write (via procfs), because a new entry
> is created in the open file table.
>
> (opening the file read-write (via fdescfs) on FreeBSD would fail)
>
> For this reason the output does not show what has been entered via
> the here-doc.
>
> In short, I was merely testing the semantics of Linux.
Ah, ok.  This is a bit of a problem on Windows.  The code tries to
reopen the file by handle.  Under some circumstances(*) we can't reopen
the file.  In that case the code just tries to duplicate the handle.
However, a duplicated file handle can't have more permissions than the
original handle.

So if it fails for you, it seems the reopen failed and the handle
only got duplicated.  In that case, you can't gain write perms if
the original handle only got read perms.

What the code fails to do is trying to open the file by name as a last
resort.  There was a (good?) reason I didn't implement that, but I don't
remember ATM.


Corinna

(*) E.g., deleted files on systems older than Windows 10 1709,
    or files on filesystems not supporting the "reopen-by-handle"
    semantics.  However, the latter case should work at least
    for non-deleted files

--
Corinna Vinschen
Cygwin Maintainer

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Houder
On Tue, 22 Jan 2019 10:41:57, Corinna Vinschen  wrote:
> On Jan 22 10:25, Houder wrote:
[snip]

> > Curious! It fails (for me) on W7 ...
>
> It works for me just as well on W7:
>
> $ uname -a
> CYGWIN_NT-6.1 vmbert764 2.12.0(0.333/5/3) 2019-01-21 22:47 x86_64 Cygwin
> $ ./stca /dev/fd/0 <<EOF
> ? bla
> ? EOF
> fd1 =3D 0
> argv[1] =3D /dev/fd/0
> fd2 =3D 3
> buf =3D \
> Hello, world!

That is odd ... (I am using the same version of Cygwin as you do, do I
not?)

64-@@ uname -a
CYGWIN_NT-6.1 Seven 2.12.0s(0.333/5/3) 2019-01-21 10:25 x86_64 Cygwin
(however, I only replaced the cygwin1.dll)

(the snapshot is apparently not the same as your version 2.12.0)

> > > Not sure what you're testing.
> >
> > STC inherits a "read-only" open file descriptor from bash. On Linux
> > the file can be opened read-write (via procfs), because a new entry
> > is created in the open file table.
> >
> > (opening the file read-write (via fdescfs) on FreeBSD would fail)
> >
> > For this reason the output does not show what has been entered via
> > the here-doc.
> >
> > In short, I was merely testing the semantics of Linux.
>
> Ah, ok.  This is a bit of a problem on Windows.  The code tries to
> reopen the file by handle.  Under some circumstances(*) we can't reopen
> the file.  In that case the code just tries to duplicate the handle.
> However, a duplicated file handle can't have more permissions than the
> original handle.
>
> So if it fails for you, it seems the reopen failed and the handle
> only got duplicated.  In that case, you can't gain write perms if
> the original handle only got read perms.

Understood.

> What the code fails to do is trying to open the file by name as a last
> resort.  There was a (good?) reason I didn't implement that, but I don't
> remember ATM.

Not relevant in this case, as the file has been deleted ... (i.e., in this
case, it cannot be opened by name).

Henri

=====


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Reply | Threaded
Open this post in threaded view
|

Re: /dev/fd/N not synonymous with file descriptor N; it is on Linux

Corinna Vinschen-2
On Jan 22 11:20, Houder wrote:

> On Tue, 22 Jan 2019 10:41:57, Corinna Vinschen  wrote:
> > On Jan 22 10:25, Houder wrote:
> [snip]
>
> > > Curious! It fails (for me) on W7 ...
> >
> > It works for me just as well on W7:
> >
> > $ uname -a
> > CYGWIN_NT-6.1 vmbert764 2.12.0(0.333/5/3) 2019-01-21 22:47 x86_64 Cygwin
> > $ ./stca /dev/fd/0 <<EOF
> > ? bla
> > ? EOF
> > fd1 =3D 0
> > argv[1] =3D /dev/fd/0
> > fd2 =3D 3
> > buf =3D \
> > Hello, world!
>
> That is odd ... (I am using the same version of Cygwin as you do, do I
> not?)
>
> 64-@@ uname -a
> CYGWIN_NT-6.1 Seven 2.12.0s(0.333/5/3) 2019-01-21 10:25 x86_64 Cygwin
> (however, I only replaced the cygwin1.dll)
Yes

> (the snapshot is apparently not the same as your version 2.12.0)

My version only differs by the fact that it's built without optimization
and with broken posix timer code since I'm revamping it ATM.

Maybe you should run the above shell session under strace and see if
something unusual crops up.  BLODA?


Corinna

--
Corinna Vinschen
Cygwin Maintainer

signature.asc (849 bytes) Download Attachment
12