[PATCH] Cygwin: console: Add code to restore console mode on close.

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[PATCH] Cygwin: console: Add code to restore console mode on close.

Takashi Yano
- The console with 24bit color support has a problem that console
  mode is changed if cygwin process is executed in cmd.exe which
  started in cygwin shell. For example, cursor keys become not
  working if bash -> cmd -> true are executed in this order.
  This patch fixes the issue.
---
 winsup/cygwin/fhandler_console.cc | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/winsup/cygwin/fhandler_console.cc b/winsup/cygwin/fhandler_console.cc
index 30b9165ca..78f42999c 100644
--- a/winsup/cygwin/fhandler_console.cc
+++ b/winsup/cygwin/fhandler_console.cc
@@ -64,6 +64,9 @@ fhandler_console::console_state NO_COPY *fhandler_console::shared_console_info;
 
 bool NO_COPY fhandler_console::invisible_console;
 
+static DWORD orig_conin_mode = (DWORD) -1;
+static DWORD orig_conout_mode = (DWORD) -1;
+
 static void
 beep ()
 {
@@ -1019,6 +1022,11 @@ fhandler_console::open (int flags, mode_t)
   get_ttyp ()->rstcons (false);
   set_open_status ();
 
+  if (orig_conin_mode == (DWORD) -1)
+    GetConsoleMode (get_handle (), &orig_conin_mode);
+  if (orig_conout_mode == (DWORD) -1)
+    GetConsoleMode (get_output_handle (), &orig_conout_mode);
+
   if (getpid () == con.owner && wincap.has_con_24bit_colors ())
     {
       DWORD dwMode;
@@ -1088,6 +1096,19 @@ fhandler_console::close ()
       SetConsoleMode (get_output_handle (), dwMode);
     }
 
+  /* Restore console mode if this is the last closure. */
+  OBJECT_BASIC_INFORMATION obi;
+  NTSTATUS status;
+  status = NtQueryObject (get_handle (), ObjectBasicInformation,
+  &obi, sizeof obi, NULL);
+  if (NT_SUCCESS (status) && obi.HandleCount == 1)
+    {
+      if (orig_conin_mode != (DWORD) -1)
+ SetConsoleMode (get_handle (), orig_conin_mode);
+      if (orig_conout_mode != (DWORD) -1)
+ SetConsoleMode (get_handle (), orig_conout_mode);
+    }
+
   CloseHandle (get_handle ());
   CloseHandle (get_output_handle ());
 
--
2.21.0

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Cygwin: console: Add code to restore console mode on close.

Corinna Vinschen-2
Hi Takashi,

On Jan  2 22:17, Takashi Yano wrote:
> - The console with 24bit color support has a problem that console
>   mode is changed if cygwin process is executed in cmd.exe which
>   started in cygwin shell. For example, cursor keys become not
>   working if bash -> cmd -> true are executed in this order.
>   This patch fixes the issue.

Is that supposed to work for deeper call trees as well?

I'm asking because I tried something like

  bash -> cmd -> bash -> cmd

and it turned out that the cursor keys don't work at all in the second
cmd, while they work fine again in the first cmd when returning to it.


Thanks,
Corinna

--
Corinna Vinschen
Cygwin Maintainer

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Cygwin: console: Add code to restore console mode on close.

Takashi Yano
Hi Corinna, welcome back!

On Mon, 13 Jan 2020 17:25:53 +0100
Corinna Vinschen wrote:
> On Jan  2 22:17, Takashi Yano wrote:
> > - The console with 24bit color support has a problem that console
> >   mode is changed if cygwin process is executed in cmd.exe which
> >   started in cygwin shell. For example, cursor keys become not
> >   working if bash -> cmd -> true are executed in this order.
> >   This patch fixes the issue.
>
> Is that supposed to work for deeper call trees as well?

I think so.

> I'm asking because I tried something like
>
>   bash -> cmd -> bash -> cmd
>
> and it turned out that the cursor keys don't work at all in the second
> cmd, while they work fine again in the first cmd when returning to it.

The cursor key itsself works in this case. You can edit command line
by left and right arrow keys. However, history does not work in this
situation. This seems to be because the history buffers are exhausted.

The same happens if you try

cmd -> cmd -> cmd -> cmd -> ........ -> cmd

in command prompt without cygwin.

I confirmed that history works in this situation if NumberOfHistoryBuffers
is increased using SetConsoleHistoryInfo(). The default value of
NumberOfHistoryBuffers seems to be 4 in windows 10.

The history buffer seems to be consumed by any process which attached to
the same console. Therefore, the same happens by

bash -> bash -> cmd

--
Takashi Yano <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Cygwin: console: Add code to restore console mode on close.

Corinna Vinschen-2
On Jan 14 10:37, Takashi Yano wrote:

> Hi Corinna, welcome back!
>
> On Mon, 13 Jan 2020 17:25:53 +0100
> Corinna Vinschen wrote:
> > On Jan  2 22:17, Takashi Yano wrote:
> > > - The console with 24bit color support has a problem that console
> > >   mode is changed if cygwin process is executed in cmd.exe which
> > >   started in cygwin shell. For example, cursor keys become not
> > >   working if bash -> cmd -> true are executed in this order.
> > >   This patch fixes the issue.
> >
> > Is that supposed to work for deeper call trees as well?
>
> I think so.
>
> > I'm asking because I tried something like
> >
> >   bash -> cmd -> bash -> cmd
> >
> > and it turned out that the cursor keys don't work at all in the second
> > cmd, while they work fine again in the first cmd when returning to it.
>
> The cursor key itsself works in this case. You can edit command line
> by left and right arrow keys. However, history does not work in this
> situation. This seems to be because the history buffers are exhausted.
>
> The same happens if you try
>
> cmd -> cmd -> cmd -> cmd -> ........ -> cmd
>
> in command prompt without cygwin.
>
> I confirmed that history works in this situation if NumberOfHistoryBuffers
> is increased using SetConsoleHistoryInfo(). The default value of
> NumberOfHistoryBuffers seems to be 4 in windows 10.
>
> The history buffer seems to be consumed by any process which attached to
> the same console. Therefore, the same happens by
>
> bash -> bash -> cmd
Ok, interesting.  I wasn't aware of that.

I pushed all of your outstanding patches now and I'll create a
developer snapshot in a bit.


Thanks,
Corinna

--
Corinna Vinschen
Cygwin Maintainer

signature.asc (849 bytes) Download Attachment