name pipe problem: 1 writer, multiple concurrent readers

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

name pipe problem: 1 writer, multiple concurrent readers

Cygwin list mailing list
Hi,

The following script:

#!/bin/sh

rm -f t.pip t.txt

mkfifo t.pip
printf "line1\nline2\n" >t.txt

ps

{
  while true ; do
    cp t.txt t.pip
  done
} &

rm -f t.rc.*
for rc in 0 1 2 3 4 5 6 7 8 9 ; do
  {
    diff t.pip t.txt
    echo $? >t.rc.$rc
  }
done
echo result1 start
ls  t.rc.* | wc -l
cat t.rc.*
echo result1 end

rm -f t.rc.*
for rc in 0 1 2 3 4 5 6 7 8 9 ; do
  {
    diff t.pip t.txt
    echo $? >t.rc.$rc
  } & # run the readers in parallel
done
sleep 10
echo result2 start
ls  t.rc.* | wc -l
cat t.rc.*
echo result2 end

ps

Give me output like this:

$ ./tpip.sh
      PID    PPID    PGID     WINPID   TTY         UID    STIME COMMAND
     1642    1600    1642      65264  cons1    1058872 13:18:58 /usr/bin/sh
     1600       1    1600      72728  cons1    1058872 13:18:33 /usr/bin/bash
     1645    1642    1642      42088  cons1    1058872 13:18:58 /usr/bin/ps
result1 start
10
0
0
0
0
0
0
0
0
0
0
result1 end
0a1,2
> line1
> line2
diff: t.pip
result2 start
1
2
result2 end
      PID    PPID    PGID     WINPID   TTY         UID    STIME COMMAND
     1690    1688    1642     104032  cons1    1058872 13:19:01 /usr/bin/diff
     1681    1642    1642      95012  cons1    1058872 13:19:00 /usr/bin/sh
     1642    1600    1642      65264  cons1    1058872 13:18:58 /usr/bin/sh
     1684    1681    1642      99624  cons1    1058872 13:19:00 /usr/bin/diff
     1678    1676    1642      94532  cons1    1058872 13:19:00 /usr/bin/diff
     1688    1642    1642      88864  cons1    1058872 13:19:01 /usr/bin/sh
     1698    1642    1642     104820  cons1    1058872 13:19:11 /usr/bin/ps
     1692    1642    1642      66572  cons1    1058872 13:19:01 /usr/bin/sh
     1677    1674    1642      86692  cons1    1058872 13:19:00 /usr/bin/diff
     1646    1642    1642      30888  cons1    1058872 13:18:58 /usr/bin/sh
     1600       1    1600      72728  cons1    1058872 13:18:33 /usr/bin/bash
     1686    1685    1642      14320  cons1    1058872 13:19:01 /usr/bin/diff
     1685    1642    1642      25608  cons1    1058872 13:19:00 /usr/bin/sh
     1676    1642    1642     104212  cons1    1058872 13:19:00 /usr/bin/sh
     1689    1642    1642      98004  cons1    1058872 13:19:01 /usr/bin/sh
     1674    1642    1642      44152  cons1    1058872 13:19:00 /usr/bin/sh
     1680    1646    1642      28224  cons1    1058872 13:19:00 /usr/bin/cp
     1682    1679    1642      43612  cons1    1058872 13:19:00 /usr/bin/diff
     1694    1692    1642      67736  cons1    1058872 13:19:01 /usr/bin/diff
     1683    1642    1642      93544  cons1    1058872 13:19:00 /usr/bin/sh
     1679    1642    1642      90188  cons1    1058872 13:19:00 /usr/bin/sh
     1691    1689    1642      68560  cons1    1058872 13:19:01 /usr/bin/diff
     1687    1683    1642      83952  cons1    1058872 13:19:01 /usr/bin/diff

$

That is, when I run the readers in sequence, no problem, but when I
run the in parallel, they either hang or t.pip seems empty.

Is this as expected?

My final goal is to have my .netrc file encrypted. t.pip would be
.netrc and t.txt would be the encrypted one. If I can get to my goal
another way, feel free to give me the idea.

/Morten
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
Reply | Threaded
Open this post in threaded view
|

Re: name pipe problem: 1 writer, multiple concurrent readers

Cygwin list mailing list
On 5/19/2020 7:26 AM, Morten Kjærulff via Cygwin wrote:

> Hi,
>
> The following script:
>
> #!/bin/sh
>
> rm -f t.pip t.txt
>
> mkfifo t.pip
> printf "line1\nline2\n" >t.txt
>
> ps
>
> {
>    while true ; do
>      cp t.txt t.pip
>    done
> } &
>
> rm -f t.rc.*
> for rc in 0 1 2 3 4 5 6 7 8 9 ; do
>    {
>      diff t.pip t.txt
>      echo $? >t.rc.$rc
>    }
> done
> echo result1 start
> ls  t.rc.* | wc -l
> cat t.rc.*
> echo result1 end
>
> rm -f t.rc.*
> for rc in 0 1 2 3 4 5 6 7 8 9 ; do
>    {
>      diff t.pip t.txt
>      echo $? >t.rc.$rc
>    } & # run the readers in parallel
> done
> sleep 10
> echo result2 start
> ls  t.rc.* | wc -l
> cat t.rc.*
> echo result2 end
>
> ps
>
> Give me output like this:
>
> $ ./tpip.sh
>        PID    PPID    PGID     WINPID   TTY         UID    STIME COMMAND
>       1642    1600    1642      65264  cons1    1058872 13:18:58 /usr/bin/sh
>       1600       1    1600      72728  cons1    1058872 13:18:33 /usr/bin/bash
>       1645    1642    1642      42088  cons1    1058872 13:18:58 /usr/bin/ps
> result1 start
> 10
> 0
> 0
> 0
> 0
> 0
> 0
> 0
> 0
> 0
> 0
> result1 end
> 0a1,2
>> line1
>> line2
> diff: t.pip
> result2 start
> 1
> 2
> result2 end
>        PID    PPID    PGID     WINPID   TTY         UID    STIME COMMAND
>       1690    1688    1642     104032  cons1    1058872 13:19:01 /usr/bin/diff
>       1681    1642    1642      95012  cons1    1058872 13:19:00 /usr/bin/sh
>       1642    1600    1642      65264  cons1    1058872 13:18:58 /usr/bin/sh
>       1684    1681    1642      99624  cons1    1058872 13:19:00 /usr/bin/diff
>       1678    1676    1642      94532  cons1    1058872 13:19:00 /usr/bin/diff
>       1688    1642    1642      88864  cons1    1058872 13:19:01 /usr/bin/sh
>       1698    1642    1642     104820  cons1    1058872 13:19:11 /usr/bin/ps
>       1692    1642    1642      66572  cons1    1058872 13:19:01 /usr/bin/sh
>       1677    1674    1642      86692  cons1    1058872 13:19:00 /usr/bin/diff
>       1646    1642    1642      30888  cons1    1058872 13:18:58 /usr/bin/sh
>       1600       1    1600      72728  cons1    1058872 13:18:33 /usr/bin/bash
>       1686    1685    1642      14320  cons1    1058872 13:19:01 /usr/bin/diff
>       1685    1642    1642      25608  cons1    1058872 13:19:00 /usr/bin/sh
>       1676    1642    1642     104212  cons1    1058872 13:19:00 /usr/bin/sh
>       1689    1642    1642      98004  cons1    1058872 13:19:01 /usr/bin/sh
>       1674    1642    1642      44152  cons1    1058872 13:19:00 /usr/bin/sh
>       1680    1646    1642      28224  cons1    1058872 13:19:00 /usr/bin/cp
>       1682    1679    1642      43612  cons1    1058872 13:19:00 /usr/bin/diff
>       1694    1692    1642      67736  cons1    1058872 13:19:01 /usr/bin/diff
>       1683    1642    1642      93544  cons1    1058872 13:19:00 /usr/bin/sh
>       1679    1642    1642      90188  cons1    1058872 13:19:00 /usr/bin/sh
>       1691    1689    1642      68560  cons1    1058872 13:19:01 /usr/bin/diff
>       1687    1683    1642      83952  cons1    1058872 13:19:01 /usr/bin/diff
>
> $
>
> That is, when I run the readers in sequence, no problem, but when I
> run the in parallel, they either hang or t.pip seems empty.

The current cygwin release doesn't support multiple concurrent readers of a
FIFO.  I have recently added that support, which should appear in the next
release, although there are still bugs to be fixed.  See

   https://cygwin.com/pipermail/cygwin-patches/2020q2/010195.html

Ken
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
Reply | Threaded
Open this post in threaded view
|

Re: name pipe problem: 1 writer, multiple concurrent readers

Cygwin list mailing list
On 5/19/2020 11:20 AM, Morten Kjærulff wrote:

> On Tue, May 19, 2020 at 3:55 PM Ken Brown via Cygwin <[hidden email]> wrote:
>>
>> On 5/19/2020 7:26 AM, Morten Kjærulff via Cygwin wrote:
>>> Hi,
>>>
>>> The following script:
>>>
>>> #!/bin/sh
>>>
>>> rm -f t.pip t.txt
>>>
>>> mkfifo t.pip
>>> printf "line1\nline2\n" >t.txt
>>>
>>> ps
>>>
>>> {
>>>     while true ; do
>>>       cp t.txt t.pip
>>>     done
>>> } &
>>>
>>> rm -f t.rc.*
>>> for rc in 0 1 2 3 4 5 6 7 8 9 ; do
>>>     {
>>>       diff t.pip t.txt
>>>       echo $? >t.rc.$rc
>>>     }
>>> done
>>> echo result1 start
>>> ls  t.rc.* | wc -l
>>> cat t.rc.*
>>> echo result1 end
>>>
>>> rm -f t.rc.*
>>> for rc in 0 1 2 3 4 5 6 7 8 9 ; do
>>>     {
>>>       diff t.pip t.txt
>>>       echo $? >t.rc.$rc
>>>     } & # run the readers in parallel
>>> done
>>> sleep 10
>>> echo result2 start
>>> ls  t.rc.* | wc -l
>>> cat t.rc.*
>>> echo result2 end
>>>
>>> ps
>>>
>>> Give me output like this:
>>>
>>> $ ./tpip.sh
>>>         PID    PPID    PGID     WINPID   TTY         UID    STIME COMMAND
>>>        1642    1600    1642      65264  cons1    1058872 13:18:58 /usr/bin/sh
>>>        1600       1    1600      72728  cons1    1058872 13:18:33 /usr/bin/bash
>>>        1645    1642    1642      42088  cons1    1058872 13:18:58 /usr/bin/ps
>>> result1 start
>>> 10
>>> 0
>>> 0
>>> 0
>>> 0
>>> 0
>>> 0
>>> 0
>>> 0
>>> 0
>>> 0
>>> result1 end
>>> 0a1,2
>>>> line1
>>>> line2
>>> diff: t.pip
>>> result2 start
>>> 1
>>> 2
>>> result2 end
>>>         PID    PPID    PGID     WINPID   TTY         UID    STIME COMMAND
>>>        1690    1688    1642     104032  cons1    1058872 13:19:01 /usr/bin/diff
>>>        1681    1642    1642      95012  cons1    1058872 13:19:00 /usr/bin/sh
>>>        1642    1600    1642      65264  cons1    1058872 13:18:58 /usr/bin/sh
>>>        1684    1681    1642      99624  cons1    1058872 13:19:00 /usr/bin/diff
>>>        1678    1676    1642      94532  cons1    1058872 13:19:00 /usr/bin/diff
>>>        1688    1642    1642      88864  cons1    1058872 13:19:01 /usr/bin/sh
>>>        1698    1642    1642     104820  cons1    1058872 13:19:11 /usr/bin/ps
>>>        1692    1642    1642      66572  cons1    1058872 13:19:01 /usr/bin/sh
>>>        1677    1674    1642      86692  cons1    1058872 13:19:00 /usr/bin/diff
>>>        1646    1642    1642      30888  cons1    1058872 13:18:58 /usr/bin/sh
>>>        1600       1    1600      72728  cons1    1058872 13:18:33 /usr/bin/bash
>>>        1686    1685    1642      14320  cons1    1058872 13:19:01 /usr/bin/diff
>>>        1685    1642    1642      25608  cons1    1058872 13:19:00 /usr/bin/sh
>>>        1676    1642    1642     104212  cons1    1058872 13:19:00 /usr/bin/sh
>>>        1689    1642    1642      98004  cons1    1058872 13:19:01 /usr/bin/sh
>>>        1674    1642    1642      44152  cons1    1058872 13:19:00 /usr/bin/sh
>>>        1680    1646    1642      28224  cons1    1058872 13:19:00 /usr/bin/cp
>>>        1682    1679    1642      43612  cons1    1058872 13:19:00 /usr/bin/diff
>>>        1694    1692    1642      67736  cons1    1058872 13:19:01 /usr/bin/diff
>>>        1683    1642    1642      93544  cons1    1058872 13:19:00 /usr/bin/sh
>>>        1679    1642    1642      90188  cons1    1058872 13:19:00 /usr/bin/sh
>>>        1691    1689    1642      68560  cons1    1058872 13:19:01 /usr/bin/diff
>>>        1687    1683    1642      83952  cons1    1058872 13:19:01 /usr/bin/diff
>>>
>>> $
>>>
>>> That is, when I run the readers in sequence, no problem, but when I
>>> run the in parallel, they either hang or t.pip seems empty.
>>
>> The current cygwin release doesn't support multiple concurrent readers of a
>> FIFO.  I have recently added that support, which should appear in the next
>> release, although there are still bugs to be fixed.  See
>>
>>     https://cygwin.com/pipermail/cygwin-patches/2020q2/010195.html
>>
>> Ken
> Great!

Let's keep the discussion on the mailing list
> Then we will have a read-only file (pipe), which is the output of an
> abitrary command, right?

Yes.

> Can we have multiple concurrent writers and one reader, which would
> become a write-only file, which becomes the input of an abitrary
> command?

That already exists, although I found and fixed some bugs when I was developing
the multiple reader support.

> Anywhere I can see the plans for next release, if any?

That's up to Corinna.

Ken
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple