I have a dream: A modern scripting language everywhere

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

I have a dream: A modern scripting language everywhere

Warren Young-2
It’s sad that we still can’t count on stronger scripting languages than POSIX shell and Awk in 2014.  We have many better languages, but none are part of POSIX, so we can’t count on having them everywhere.

Perl and Python are probably too big for The Open Group to make one of them a requirement for receiving the right to use the Unix trademark.  I don’t see another organization that could move all *ix-likes to adopt such a language.

If the standards groups won’t do it, that leaves us with de facto standards. Red Hat and Google are trying to push Python for this, but I don’t think either has enough power to move competing organizations on this.  And again, Python is too big to be part of a “small parts, loosely joined” OS definition.

Lua is small and liberally-licensed, but a bit too impoverished in its stock form to be an effective improvement on the current state.

FSF failed to push Scheme/Guile into this role, for which I give thanks.

That’s part of the problem, of course: getting a bunch of fractious geeks to agree on a syntax.  Awk and Ksh got into POSIX when AT&T + Sun had the combined might to force them through the process.  The *ix world is too diffuse now for that.

I love the Javascript dialect (!) of Scheme.  Say what you want against JS; it could be better, but those who say it’s the worst language on the planet are either ignorant or being hyperbolic.

But, fan though I am, I’ll quickly acknowledge that Javascript isn’t going to solve this problem any time soon, either.  The universality of the web has pushed a JS interpreter onto pretty much every computing device big enough to have a keyboard, but the only hope for turning it into an effective Unix scripting language is coming via node.js, and they’re years away from adding enough library support to JS to make it fit for such a battle.

I think only Ruby is powerful enough, small enough, and non-controversial enough to solve this.  It’s Perlish and Pythonic and Scheme-like enough to have wide appeal and applicability.  I’m not a Ruby fan.  I came up via Perl, and Ruby never struck me as different enough to be worth switching.  But, if I could count on Ruby everywhere, that would change things.

Swift would be a nice solution, but it’ll never really escape the confines of the Apple world.  The same goes for any of the .NET languages, even with all this new openness.

Anyone else see a path out of this situation?
Reply | Threaded
Open this post in threaded view
|

Re: I have a dream: A modern scripting language everywhere

Corinna Vinschen-2
On Dec  8 15:59, Warren Young wrote:

> It’s sad that we still can’t count on stronger scripting languages
> than POSIX shell and Awk in 2014.  We have many better languages, but
> none are part of POSIX, so we can’t count on having them everywhere.
>
> Perl and Python are probably too big
> [...]
> I love the Javascript dialect (!) of Scheme.  Say what you want
> against JS; it could be better, but those who say it’s the worst
> language on the planet are either ignorant or being hyperbolic.
>
> But, fan though I am, I’ll quickly acknowledge that Javascript isn’t
> going to solve this problem any time soon, either.  The universality
> of the web has pushed a JS interpreter onto pretty much every
> computing device big enough to have a keyboard, but the only hope for
> turning it into an effective Unix scripting language is coming via
> node.js, and they’re years away from adding enough library support to
> JS to make it fit for such a battle.
>
> I think only Ruby is powerful enough, small enough, and
> non-controversial enough to solve this.  It’s Perlish and Pythonic and
> Scheme-like enough to have wide appeal and applicability.  I’m not a
> Ruby fan.  I came up via Perl, and Ruby never struck me as different
> enough to be worth switching.  But, if I could count on Ruby
> everywhere, that would change things.
>
> Swift would be a nice solution, but it’ll never really escape the
> confines of the Apple world.  The same goes for any of the .NET
> languages, even with all this new openness.
>
> Anyone else see a path out of this situation?
The neat thing with shell scripting is that it's relatively easy and by
using tools like awk, sed, etc easy to extend in functionality(*).
Everybody can cobble together simple scripts, and over time everybody
can cobble together more complex scripts.  The complexitiy and inhibition
level to learn a complex scripting language is higher, I think.

But the worst problem with non-shell scripting languages is that you
introduce yet another level of "library" functions.  That's the biggest
difference between shell scripting and scripting in "high-level"
scripting languages:

(*) Shell scripts are basically the shell (pardon the pun) about the
soft bits, collected from other standarized tools.  With the help
of dozen small helpers, io-redirection and variable substitution you
can create the most complex scripts.  However, the shell itself never
requires "extensions".  No "plugins".

High-level scripting languages have to provide all this stuff by
themselves.  They either have a mighty library, or they have to allow to
create new libraries, or they have a plugin mechanism.  But there will
always be yet another functionality missing, so the thing grows and
grows and nobody is able to keep track what you can do and what you
can't, and how, and in how many variations you can do the same.

And then, years after creating all this stuff, a couple of core
developers are really unhappy with what they created because it's
missing some important feature, or it's too slow, or whatever, and they
start to create "version 2" of their scripting language, deliberately or
accidentally breaking backward compatibility with the already given
ecosystem.  It's a mess.

This will (hopefully) never happen with shell scripting.  And this is
why shell scripting prevails.  IMHO.


Corinna

attachment0 (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: I have a dream: A modern scripting language everywhere

Warren Young-2
On Dec 9, 2014, at 3:32 AM, Corinna Vinschen <[hidden email]> wrote:

> On Dec  8 15:59, Warren Young wrote:
>>
>> Anyone else see a path out of this situation?
>
> The neat thing with shell scripting is that it's relatively easy and by
> using tools like awk, sed, etc easy to extend in functionality(*).

True, but that’s also the biggest weakness of shell scripting: it takes a fork() to get almost anything of consequence done.

It was seen as a big advance when we got internal arithmetic in the shell instead of having to call out to bc.  Arithmetic is one of the core competencies of electronic computers going back to Konrad Zuse, yet shell scripts couldn’t do it internally until, what, ksh93? [1]

Mind, I do a lot of shell scripting, and I’m happy to have it, relative to the situation in stock Windows.

The POSIX shell environment might actually be the biggest draw of Cygwin for me.  Some might say it’s the package repository instead, but I suspect most Cygwin users are installing a relatively small set of non-Base packages.

But let’s get back on topic:

I don’t believe we would have kSLOC scale shell scripts if we had something more capable to write highly-portable programs in, which can sit in the gap between sh and C.

Imagine how much nicer the Autotools would be if they could count on something more powerful than POSIX.1.  It’s a big enough problem that it’s been independently re-solved several times, in unsatisfactory ways:

- CMake and Jam are written in C, so you have to build a separate tool before you can build the program you actually wanted, rather than just run a script under an interpreter already on the system.

- SCons and Bakefile require Python, and Ant requires Java.

None of these tools behave like the Autotools, just writing out a platform-independent set of scripts that run pretty much everywhere, except for Bakefile, which is built on top of Autotools.

> The complexitiy and inhibition
> level to learn a complex scripting language is higher, I think.

The gap between Bash and Perl/Python/Ruby is indeed large, but not so large as the gap between Bash and C.

> But the worst problem with non-shell scripting languages is that you
> introduce yet another level of "library" functions.

This is part of what I was getting at with my “too big” observations.

This cannot be a fatal flaw, though.  The success of C and C++ prove that it is possible to have a sincerely standardized language with a large-ish library.

A standardization effort focused on increasing the likelihood of widespread reimplementation to near-certainty with an expectation of high compatibility among implementations would have to keep a lid on the scope of the library, just as has been done for C and C++.

If Perl, Python or Ruby went through such a process, I’d expect them to leave quite a lot of the current “standard” library out of the formal standard.  This can be done successfully, as with .NET Micro Framework and J2ME. [2][3]

Just to pick on one random example, it wouldn’t be reasonable to expect Ruby’s gdbm module to survive a standardization effort.  Otherwise, even Ruby’s standard library is too big to expect anyone else to reimplement it.[4]

I carefully qualified the term “standard” above because I want to sweep aside such “standards” as the current .NET one, which is little more than a formal effort to document whatever Visual Studio happened to be able to compile at the time they started work.  We have Mono *despite* the ECMA and ISO standards, not as a result.

It’s looking like .NET is going to follow the Perl/Lua route.  Microsoft recently released enough of .NET under a liberal enough license that we may end up with just one widely-ported open source implementation, rather than a true standard in the C/sh sense.[5]

Python does have a bunch of independent reimplementations of the core language.  (PyPy, Iron Python, Jython…)  But, the “batteries included” philosophy of the library again puts it into the “too big” camp, unless you prune it way back as part of standardization.

Javascript and Scheme are both sincere standards in the C/sh sense, and thus have been widely reimplemented, with high levels of cross-compatibility.  Lua could be in the same camp, but the MIT license it’s released under makes formal standardization and reimplementation pointless.  These three are all *too* small to solve our problem here: they’d have to be considerably extended to compete with shell scripting.

> the shell itself never
> requires "extensions".  No "plugins”.

POSIX.1 is the shell’s standard library.

I don’t have to tell you that it’s pretty big. :)

> And then, years after creating all this stuff, a couple of core
> developers are really unhappy with what they created because it's
> missing some important feature, or it's too slow, or whatever, and they
> start to create "version 2" of their scripting language, deliberately or
> accidentally breaking backward compatibility with the already given
> ecosystem.  It's a mess.

This certainly does happen, but observe that D and Objective C have so far failed to kill off C and C++.

Also observe the resilience of Perl 5 and Python 2.  “New and better,” is correctly seen here as “breaks all my existing code.”



[1] Is that not where $((a+b)) and typeset -i came from?

[2] http://en.wikipedia.org/wiki/.NET_Micro_Framework

[3] http://en.wikipedia.org/wiki/Java_Platform,_Micro_Edition

[4] http://www.ruby-doc.org/stdlib-2.1.5/

[5] http://blogs.msdn.com/b/dotnet/archive/2014/11/12/net-core-is-open-source.aspx

Reply | Threaded
Open this post in threaded view
|

Re: I have a dream: A modern scripting language everywhere

Corinna Vinschen-2
On Dec  9 14:07, Warren Young wrote:

> On Dec 9, 2014, at 3:32 AM, Corinna Vinschen <[hidden email]> wrote:
> > On Dec  8 15:59, Warren Young wrote:
> >>
> >> Anyone else see a path out of this situation?
> >
> > The neat thing with shell scripting is that it's relatively easy and by
> > using tools like awk, sed, etc easy to extend in functionality(*).
>
> True, but that’s also the biggest weakness of shell scripting: it
> takes a fork() to get almost anything of consequence done.
Clean separation of tasks.  If the task crashes it doesn't crash the
shell.

> It was seen as a big advance when we got internal arithmetic in the
> shell instead of having to call out to bc.  Arithmetic is one of the
> core competencies of electronic computers going back to Konrad Zuse,
> yet shell scripts couldn’t do it internally until, what, ksh93? [1]

Well, yes.  And I admit freely that I'm using bash extensions.  Bash's
variable substitutions allow to create scripts calling cut, sed or grep
a lot less.

> Mind, I do a lot of shell scripting, and I’m happy to have it,
> relative to the situation in stock Windows.
>
> The POSIX shell environment might actually be the biggest draw of
> Cygwin for me.  Some might say it’s the package repository instead,
> but I suspect most Cygwin users are installing a relatively small set
> of non-Base packages.

Just that everybody installs something else.

> But let’s get back on topic:
>
> I don’t believe we would have kSLOC scale shell scripts if we had
> something more capable to write highly-portable programs in, which can
> sit in the gap between sh and C.

If you look close enough, you can always find a gap.  To me, the only
reason to see a gap between sh and C is my own laziness.  Sometimes I'd
like to try something without having to *think* about coding up in
either shell or C.  However, the gap-filling stuff *still* requires
thinking.  There's not a lot of tools which solve your problem right
there, on the spot, with a miracle commandline option.

> [...]
> > The complexitiy and inhibition
> > level to learn a complex scripting language is higher, I think.
>
> The gap between Bash and Perl/Python/Ruby is indeed large, but not so
> large as the gap between Bash and C.

Au contraire, mon capitan.  The complexity in C vs. P/P/R is not in the
language.  Take a look into any one perl script not written by yourself.
The *only* problem of C vs. scripting languages is... the compiler.
C isn't interpreted.  Thus you can't just run C code, you have to
compile it.  So, to learn C, you also have to learn using build tools.
This is the double step you have to take to learn any compiled programming
language and that's what keeps people shying away, even if they know
the ins and outs of some script language.

> > But the worst problem with non-shell scripting languages is that you
> > introduce yet another level of "library" functions.
>
> This is part of what I was getting at with my “too big” observations.
>
> This cannot be a fatal flaw, though.  The success of C and C++ prove
> that it is possible to have a sincerely standardized language with a
> large-ish library.

Yes, but this is the standard library.  This is the OS-specifc part.
This is what you need to know to implement the other.  This is the
core of programming and no scripting langauge works without it.

Scripting languages add another layer on top of that.  That's what
I was talking about.  YA layer of libs, and plugins, and plugins
for plugins.

> If Perl, Python or Ruby went through such a process, I’d expect them
> to leave quite a lot of the current “standard” library out of the
> formal standard.  This can be done successfully, as with .NET Micro
> Framework and J2ME. [2][3]
>
> Just to pick on one random example, it wouldn’t be reasonable to
> expect Ruby’s gdbm module to survive a standardization effort.
> Otherwise, even Ruby’s standard library is too big to expect anyone
> else to reimplement it.[4]

Absolutely.

> [...]
> > the shell itself never
> > requires "extensions".  No "plugins”.
>
> POSIX.1 is the shell’s standard library.
>
> I don’t have to tell you that it’s pretty big. :)

LOL.  You're redefining for the sake of argument.  But, yes, in a sense,
this is a "standard library".

> > And then, years after creating all this stuff, a couple of core
> > developers are really unhappy with what they created because it's
> > missing some important feature, or it's too slow, or whatever, and they
> > start to create "version 2" of their scripting language, deliberately or
> > accidentally breaking backward compatibility with the already given
> > ecosystem.  It's a mess.
>
> This certainly does happen, but observe that D and Objective C have so
> far failed to kill off C and C++.

You can't kill C ;)

> Also observe the resilience of Perl 5 and Python 2.  “New and better,”
> is correctly seen here as “breaks all my existing code.”

Which is good, I guess.  Cgf and I often discussed if Cygwin should move
to a complete reimplementaion "Cygwin 2", but in the end we'd probably
have had the same problems as perl 5.


Corinna

attachment0 (836 bytes) Download Attachment