wildcards don't work in directory with files with odd characters

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

wildcards don't work in directory with files with odd characters

jeffunit
I have a directory that has some files with odd files.
I can do a 'ls', successfully. However if I do a 'ls *'' I get:
ls: cannot access '*': No such file or directory

Here is ls output:
'Highlander-S03E21-Final'$'\303\251''_Part_I-22.mkv'
'Highlander-S03E22-Final'$'\303\251''_Part_II-23.mkv'

I am pretty sure this used to work.
This is not specific to ls. wc has the same behavior for example.

cygcheck.out is enclosed.

thanks,
jeff


--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

cygcheck.out (29K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: wildcards don't work in directory with files with odd characters

Andrey Repin
Greetings, jeff!

> I have a directory that has some files with odd files.
> I can do a 'ls', successfully. However if I do a 'ls *'' I get:
> ls: cannot access '*': No such file or directory

> Here is ls output:
> 'Highlander-S03E21-Final'$'\303\251''_Part_I-22.mkv'
> 'Highlander-S03E22-Final'$'\303\251''_Part_II-23.mkv'

> I am pretty sure this used to work.
> This is not specific to ls. wc has the same behavior for example.

Are you trying to run it from Cygwin shell or from some native one, like cmd?

> cygcheck.out is enclosed.


--
With best regards,
Andrey Repin
Monday, May 18, 2020 18:55:16

Sorry for my terrible english...

--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
Reply | Threaded
Open this post in threaded view
|

Re: wildcards don't work in directory with files with odd characters

jeffunit
On 5/18/2020 8:55 AM, Andrey Repin wrote:

> Greetings, jeff!
>
>> I have a directory that has some files with odd files.
>> I can do a 'ls', successfully. However if I do a 'ls *'' I get:
>> ls: cannot access '*': No such file or directory
>> Here is ls output:
>> 'Highlander-S03E21-Final'$'\303\251''_Part_I-22.mkv'
>> 'Highlander-S03E22-Final'$'\303\251''_Part_II-23.mkv'
>> I am pretty sure this used to work.
>> This is not specific to ls. wc has the same behavior for example.
> Are you trying to run it from Cygwin shell or from some native one, like cmd?
>
I am running from windows 'command prompt' aka cmd. When run from bash
everything seems to work correctly.

--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
Reply | Threaded
Open this post in threaded view
|

Re: wildcards don't work in directory with files with odd characters

Adam Dinwoodie-2
On Mon, 18 May 2020 at 17:21, jeff wrote:

>
> On 5/18/2020 8:55 AM, Andrey Repin wrote:
> > Greetings, jeff!
> >
> >> I have a directory that has some files with odd files.
> >> I can do a 'ls', successfully. However if I do a 'ls *'' I get:
> >> ls: cannot access '*': No such file or directory
> >> Here is ls output:
> >> 'Highlander-S03E21-Final'$'\303\251''_Part_I-22.mkv'
> >> 'Highlander-S03E22-Final'$'\303\251''_Part_II-23.mkv'
> >> I am pretty sure this used to work.
> >> This is not specific to ls. wc has the same behavior for example.
> > Are you trying to run it from Cygwin shell or from some native one, like cmd?
> >
> I am running from windows 'command prompt' aka cmd. When run from bash
> everything seems to work correctly.

In which case this is expected behaviour: Cygwin's `ls` expects the
shell (e.g. Bash) to expand globs like `*`, but Windows' command
prompt expects applications to handle expanding globs (or the Windows
equivalents thereof) themselves. When you call a Cygwin command like
`ls` directly from the Windows command prompt, Windows passes the
arguments as-is to the Cygwin command, and the Cygwin command assumes
that the arguments it received are already appropriately expanded.

If this was working previously, I can only assume it's because you
were calling Windows' `ls` (which I seem to recall exists and is
essentially an alias for `dir`), which expects Windows semantics and
therefore handles its own expansions.

HTH

Adam
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
Reply | Threaded
Open this post in threaded view
|

Re: wildcards don't work in directory with files with odd characters

jeffunit
On 5/18/2020 11:03 AM, Adam Dinwoodie wrote:

> On Mon, 18 May 2020 at 17:21, jeff wrote:
>> On 5/18/2020 8:55 AM, Andrey Repin wrote:
>>> Greetings, jeff!
>>>
>>>> I have a directory that has some files with odd files.
>>>> I can do a 'ls', successfully. However if I do a 'ls *'' I get:
>>>> ls: cannot access '*': No such file or directory
>>>> Here is ls output:
>>>> 'Highlander-S03E21-Final'$'\303\251''_Part_I-22.mkv'
>>>> 'Highlander-S03E22-Final'$'\303\251''_Part_II-23.mkv'
>>>> I am pretty sure this used to work.
>>>> This is not specific to ls. wc has the same behavior for example.
>>> Are you trying to run it from Cygwin shell or from some native one, like cmd?
>>>
>> I am running from windows 'command prompt' aka cmd. When run from bash
>> everything seems to work correctly.
> In which case this is expected behaviour: Cygwin's `ls` expects the
> shell (e.g. Bash) to expand globs like `*`, but Windows' command
> prompt expects applications to handle expanding globs (or the Windows
> equivalents thereof) themselves. When you call a Cygwin command like
> `ls` directly from the Windows command prompt, Windows passes the
> arguments as-is to the Cygwin command, and the Cygwin command assumes
> that the arguments it received are already appropriately expanded.
>
> If this was working previously, I can only assume it's because you
> were calling Windows' `ls` (which I seem to recall exists and is
> essentially an alias for `dir`), which expects Windows semantics and
> therefore handles its own expansions.
If the directory doesn't contain any files with odd characters in the
name, then
'ls *' run from cmd works just fine. Only when there are odd characters
in the file name is there an issue.

thanks,
jeff


--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
Reply | Threaded
Open this post in threaded view
|

Re: wildcards don't work in directory with files with odd characters

Cygwin list mailing list
In reply to this post by Adam Dinwoodie-2
On Mon, May 18, 2020 at 8:07 PM Adam Dinwoodie <[hidden email]> wrote:
> Cygwin's `ls` expects the
> shell (e.g. Bash) to expand globs like `*`, but Windows' command
> prompt expects applications to handle expanding globs (or the Windows
> equivalents thereof) themselves. When you call a Cygwin command like
> `ls` directly from the Windows command prompt, Windows passes the
> arguments as-is to the Cygwin command, and the Cygwin command assumes
> that the arguments it received are already appropriately expanded.

This is actually false.  The official FAQ mentions it as well
here<https://cygwin.com/faq.html#faq.api.globbing>.

I went on to investigate what's the issue, but I can't replicate it.
Things like the console code page, the system code
page<https://www.digitalcitizen.life/changing-display-language-used-non-unicode-programs>
could be at play, but I don't see why it would behave like this.

--
David Macek
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
Reply | Threaded
Open this post in threaded view
|

Re: wildcards don't work in directory with files with odd characters

jeffunit
On 5/18/2020 1:01 PM, David Macek via Cygwin wrote:

> On Mon, May 18, 2020 at 8:07 PM Adam Dinwoodie <[hidden email]> wrote:
>> Cygwin's `ls` expects the
>> shell (e.g. Bash) to expand globs like `*`, but Windows' command
>> prompt expects applications to handle expanding globs (or the Windows
>> equivalents thereof) themselves. When you call a Cygwin command like
>> `ls` directly from the Windows command prompt, Windows passes the
>> arguments as-is to the Cygwin command, and the Cygwin command assumes
>> that the arguments it received are already appropriately expanded.
> This is actually false.  The official FAQ mentions it as well
> here<https://cygwin.com/faq.html#faq.api.globbing>.
>
> I went on to investigate what's the issue, but I can't replicate it.
> Things like the console code page, the system code
> page<https://www.digitalcitizen.life/changing-display-language-used-non-unicode-programs>
> could be at play, but I don't see why it would behave like this.
>
I can make a small tar file with some files with odd names if that would
help.

jeff

--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
Reply | Threaded
Open this post in threaded view
|

Re: wildcards don't work in directory with files with odd characters

Chris Wagner
In reply to this post by jeffunit
On 2020-05-18 8:40 am, jeff wrote:
> I have a directory that has some files with odd files.
> I can do a 'ls', successfully. However if I do a 'ls *'' I get:
> ls: cannot access '*': No such file or directory
>
> Here is ls output:
> 'Highlander-S03E21-Final'$'\303\251''_Part_I-22.mkv'
> 'Highlander-S03E22-Final'$'\303\251''_Part_II-23.mkv'


I can confirm this.  I've seen weirdo behavior before with Cygwin
commands having to interpret wild cards internally.  It's probably
related to some wonky unicode optimization or incomplete wildcard
implementation.  This falls under the category of "don't do that". :)


Mintty:
$ uname -a; cygcheck -c coreutils
CYGWIN_NT-6.1 applejack 3.1.0(0.340/5/3) 2019-11-19 13:58 x86_64 Cygwin
Cygwin Package Information
Package              Version        Status
coreutils            8.26-2         OK

wagnerc@applejack /tmp/wild
$ touch 'Highlander-S03E21-Final'$'\303\251''_Part_I-22.mkv'; dir *
-rw-rw-r--+ 1 0 May 18 21:46 foo
-rw-rw-r--+ 1 0 May 18 21:55 Highlander-S03E21-Finalé_Part_I-22.mkv


CMD:
C:\cygwin64\tmp\wild>dir
  Volume in drive C is INTELWINNT
  Volume Serial Number is 642B-E7BA

  Directory of C:\cygwin64\tmp\wild

05/18/2020  09:46 PM    <DIR>          .
05/18/2020  09:46 PM    <DIR>          ..
05/18/2020  09:46 PM                 0 foo
05/18/2020  09:55 PM                 0
Highlander-S03E21-Finalé_Part_I-22.mkv
                2 File(s)              0 bytes
                2 Dir(s)  35,887,038,464 bytes free

C:\cygwin64\tmp\wild>ls
'Highlander-S03E21-Final'$'\303\251''_Part_I-22.mkv'   foo

C:\cygwin64\tmp\wild>ls *
ls: cannot access '*': No such file or directory

C:\cygwin64\tmp\wild>ls f*
foo

C:\cygwin64\tmp\wild>ls H*
ls: cannot access 'H*': No such file or directory



Thanks.





--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
Reply | Threaded
Open this post in threaded view
|

Re: wildcards don't work in directory with files with odd characters

Andrey Repin
In reply to this post by jeffunit
Greetings, jeff!

>>> I have a directory that has some files with odd files.
>>> I can do a 'ls', successfully. However if I do a 'ls *'' I get:
>>> ls: cannot access '*': No such file or directory
>>> Here is ls output:
>>> 'Highlander-S03E21-Final'$'\303\251''_Part_I-22.mkv'
>>> 'Highlander-S03E22-Final'$'\303\251''_Part_II-23.mkv'
>>> I am pretty sure this used to work.
>>> This is not specific to ls. wc has the same behavior for example.
>> Are you trying to run it from Cygwin shell or from some native one, like cmd?
>>
> I am running from windows 'command prompt' aka cmd. When run from bash
> everything seems to work correctly.

Then take a look at this thread:
https://cygwin.com/pipermail/cygwin/2020-March/thread.html#174

You'll need correctly configured LANG and CYGWIN's glob option.


--
With best regards,
Andrey Repin
Tuesday, May 19, 2020 13:48:25

Sorry for my terrible english...

--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
Reply | Threaded
Open this post in threaded view
|

Re: wildcards don't work in directory with files with odd characters

Chris Wagner
On 2020-05-19 6:49 am, Andrey Repin wrote:
> Then take a look at this thread:
> https://cygwin.com/pipermail/cygwin/2020-March/thread.html#174
>
> You'll need correctly configured LANG and CYGWIN's glob option.

That info should definately be in the FAQ.  Although I would still
consider this behavior a bug.


C:\cygwin64\tmp\wild>dir
05/18/2020  09:46 PM                 0 foo
05/18/2020  09:55 PM                 0
Highlander-S03E21-Finalé_Part_I-22.mkv

C:\cygwin64\tmp\wild>set LANG=
C:\cygwin64\tmp\wild>ls *
ls: cannot access '*': No such file or directory

C:\cygwin64\tmp\wild>set LANG=en_US
C:\cygwin64\tmp\wild>ls *
foo  Highlander-S03E21-Finalé_Part_I-22.mkv

C:\cygwin64\tmp\wild>set LANG=x
C:\cygwin64\tmp\wild>ls *
ls: cannot access '*': No such file or directory

C:\cygwin64\tmp\wild>set LANG=en
C:\cygwin64\tmp\wild>ls *
foo

C:\cygwin64\tmp\wild>set LANG=de
C:\cygwin64\tmp\wild>ls *
foo

C:\cygwin64\tmp\wild>set LANG=en_US
C:\cygwin64\tmp\wild>ls *
foo  Highlander-S03E21-Finalé_Part_I-22.mkv



Thanks.
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple