2015.15: We figured out the truth about “0”!

Jonathan Workington

jnthn just published a new blog post last night describing the fund that was recently set up for Perl 6 development (thank you, WenZPerl!) that’ll be used among other things for funding more jnthn time until the release coming in december. There’s also a lot of information on what jnthn has recently been doing in that blog post:

  • Unicode database in MoarVM updated to version 7.0
  • unsigned types in NativeCall (already mentioned in the previous p6weekly)
  • a whole lot of RT tickets fixed, like improved error messages for BEGIN-time things and constants, “where” constraints on attributes and variables
  • the beginning of the Uni class and its features, like different normalization forms (as specified by Unicode)

Performance improvements

Nicholas Clark has been churning out patches to reduce the number of cpu instructions “used up” during rakudo’s startup and also improved the efficiency of our serialization format.

TimToady improved the MATCH method in both rakudo and nqp that are responsible for creating the high-level object that gets created for a regexp match. That’s quite a hot method and initial, noisy measurements showed it shaves off about 5% of the time spent in the “stage parse” of rakudo’s core setting.

In addition to MATCH being faster now, rakudo (and nqp) used to flip ginormous amounts of string data whenever it encountered a <?after …>, but now [correction:] if the length of the pattern inside is known, it turns into a <?before …> with the pattern itself flipped.

I’ve built support for a whole bunch of reprops (push/pop, elems, access to attributes, keys or positions, …) into the JIT for the cases where we couldn’t generate a super simple piece of assembly code to handle something (like pushing into a list that may have to enlarge the list) but we still know what exact function we’ll be calling. Unfortunately my desktop is currently not fully assembled, otherwise i’d have some benchmark data for you.

labster gave Str its own “chars” method (as opposed to inheriting it from Cool), which makes .chars on a String work about 40% faster. Wow!

Semantic improvements

Something that may have surprised a lot of people (myself included) was that Perl 6 claims that the truth value of “0” ought to be False. That is no longer the case, and so the rule for the truth value of strings is now just “does it have any characters in it?”. I think that’s more sensible.

TimToady improved the parser’s handling of heredocs in constant expressions and BEGIN blocks. There’s a bit of cheating necessary in the parser to make all of this work, but masak wrote a post on his blog about this very topic that I recommend you read.

TimToady also implemented Complex literals inside < >, which means that <3+2i>.WHAT now properly results in a Complex, rather than a Str.

Another thing TimToady did was make the “…” operator (for smart sequences) behave more correctly with regards to strings.

Thanks to lizmat, the say and note methods and subs now properly respect the “nl” setting on the IO object used.

Another thing I ought to mention is the stellar ticket and testing work that bartolin has been doing for what feels like many months.

Documentation

There’s now a web site examples.perl6.org with lots of example code for you to peruse!

On the doc.perl6.org site, there’s now language documentation about control flow statements as well as exceptions.

Parting words

One other thing worth mentioning is that you now get a perl6-gdb-m and perl6-valgrind-m script installed with your rakudo for quickly running your stuff under gdb or valgrind.

And with that I’m already finished with the weekly. This time it’s done before midnight local time, too!

I wish you a pleasant week; until next time!

2015.14: Happy easter, enjoy your unsigned types!

Please excuse the delay; turns out the Easter weekend was a bit more busy than I expected it to be, and getting the post out on tuesday wasn’t so successful, either. I hope yours was just as nice as mine, in any case :)

I’ll report the changes from the perspective of monday (monyay!) anyway, since otherwise the next weekly might end up looking quite meager ;)

There’s already a few nice things I could have reported post-monday, so stay tuned for next week!

NativeCall improvements and unsignedness all around

NativeCall used to give errors when it found things like “uint” or “uint8″ in its argument lists. That has been fixed on both MoarVM and the JVM! For some reason, int64 was also not supported in NativeCall, but it now is. Also, why not use “longlong” or “ulonglong” every now and then? ;)

In other NativeCall news, parameters marked “is rw” (for int and num types) now work! You can have a look at the test file and the corresponding .c file to get a feel for how this works.

We’ll catch you when you fall …

There’s a few more cases where forgetting you’re writing Perl 6 code is gracefully caught by the compiler. For example, starting a file with a “package” declaration, or writing “elseif” or “else if” or “elif”. Now there’s “commatraps” where perl5 allowed writing something like “map { $_ + 5 } 1, 2, 3″ where there’s no need to put a comma after the block. Rakudo will remind you of the “Missing comma after block argument to map” if you write code like that.

In addition, something that comes up somewhat often is confusing the Whatever Star with a simple $_. Writing code like “map { * + 5 }, 1, 2, 3″ results in a list of three WhateverCode objects, because “* + 5″ already means something like “{ $_ + 5 }”, thus turning the whole thing into basically “map { { $_ + 5 } }, 1, 2, 3″. The current error message for that reads “Malformed double closure; WhateverCode is already a closure without curlies, so either remove the curlies or use valid parameter syntax instead of *”. A bit wordy, but I think it helps you towards fixing your code.

Another new error-related feature is that if you get a “Missing or wrong version of dependency of …” error, it’ll now also tell you what exact module had the dependency on the missing thing. This’ll make it much easier to track what you need to recompile/reinstall in order to get things back into working order.

A listy change

TimToady has started “undistinguishing” Parcel and List, so that the two types can become one for the Great List Refactor.

In case you don’t remember, the Great List Refactor is going to make lists (and lists of lists, and itemized lists) and flattening behavior of subs, methods and operators more uniform and easy to remember, but also improve the performance of things like list iteration a whole lot – currently there’s a whole lot of administrative overhead involved in for loops and such …

A performance hit

Sadly, merging the S11 changes that introduce a more database-like way of installing and storing packages made startup quite a bit slower. That’s because we now have a JSON “database” that we have to parse during startup. Sadly, the performance of JSON::Tiny (which is pretty much exactly what we use in CompUnitRepo::Installation) is not terribly good. In the future, the “database” will most likely be precompiled and loading time should be drastically better.

Miscellaneous stuff

There’s of course a few performance improvements again, but nothing mind-blowing this time.

On the ecosystem side of things, there’s a few new modules:

  • overwatch monitors your program’s dependencies and restarts the process if they have changed
  • Pastebin::Shadowcat posts to and retrieves pastes from the pastebin hosted on scsys.co.uk
  • App::Kains is a tool to constrain programs to a namespace-based container on linux
  • EventEmitter adds a coating of sugar on top of Supply to give a subscription interface to any object that allows arbitrary named events to be used

I think that’s most of the things that have happened over the past week; Since I’m already quite late and also running out of energy a little bit, I’ll just end it here. Feel free to leave anything I’ve missed in the comments section below :)

2015.13: The first weekly of 2015 that won’t be mistaken for a monthly!

Hi! This week marks the first week this year where the week number (as per ISO something or other) that’s bigger than 12. A few weeks ago, I got up in the middle of the night and read on the IRC that someone was surprised about the number of the weekly i had just released before going to bed. I was so surprised (and still tired) that i immediately “fixed” the number to what the current month was. Silly me!

Anyway, here’s what’s new in the land of the perls and moars:

REPL improvements

hoelzro has been working on removing linenoise – the readline replacement we’ve been using in MoarVM that’s been a bit troublesome with regards to windows support and multi-byte characters – from MoarVM core and re-introducing it as a Perl 6 module, this time even with tab completion. In addition to that, psch has been poking at making the REPL understand when a statement has only been partially entered and when it should ask for more text to finish the statement. This will make using the REPL so much more comfortable once it’s landed!

Properness improvements

TimToady has spent some time deep in the grammar and a few operators/metaoperators here and there to make parsing more robust and correct, give better errors (or get us back to previously good error messages in regressed cases) and also made the reduction meta-operator, when applied to a list infix metaoperator like Z or X able to work properly with lists of lists, even when they are specified indirectly.

jnthn just fixed a bunch of problems with \( … ) syntax for Captures. It now accepts ; and differentiates properly between Pair objects and named parameters and also accepts flattening into the Capture with the | prefix operator.

jnthn also advanced the state of native arrays and their serialization (which mostly causes pre-compiled modules to work better) on both MoarVM and JVM. Serialization of parameterized types is also partially in now, but interning those types isn’t implemented yet.

Staying on the MoarVM side of things, rudi_s appeared on the IRC channel and dropped off a bunch of patches to make our code nicer: unused functions and variables were thrown out, functions that don’t return or where the result ends up unused were properly marked, format strings were fixed. I don’t know what rudi’s involvement will end up being, but I hope when the codebase feels more familiar to them, they’ll make some deeper changes :)

Performance improvements

While working on making “use nqp” a mandatory declaration for using nqp:: ops in modules, moritz found out that “nqp::time_n” is a very popular op across our ecosystem. Why is that, you ask? Because “now”, which gives you a posix timestamp with fractions-of-seconds accuracy, is terribly slow. Actually, I should say “was” rather than “is”. jnthn found out that a big chunk of the time spent in “now” comes from the coercer that turns real values (Num) into rational values (Rat). That thing is now about 6.5x faster than it used to be.

jnthn also had a close look at how spesh behaves during startup and tuned the specializer and on-stack-replacement thresholds in MoarVM to reduce overhead during startup. This impacts both memory usage and time usage.

The Ecosystem

In the ecosystem there’s now the beginnings of a Git::PurePerl port from p5 to p6, which can already clone and checkout repositories, albeit very slowly. During the week two other modules showed up, namely Linux::Fuser and Browser::Open.

Another interesting thing to point out is that thanks to a bit of work by retupmoca GTK::Simple will now download DLL files from gtk-dlls.p6c.org if they can’t be found on a windows system. This makes deploying GTK-using Perl 6 programs to windows systems much simpler. I already asked retupmoca to apply the same treatment to a few more modules.

I personally spent a bit of time playing around with HTTP::Server::Async and i’ve learned a thing or two about javascript, HTML5 technologies (in particular EventSource, which is kinda cool) and how to glue it together with Perl 6. A few things have come up that ought to be changed in HTTP::Server::Async, but tony-o (HTTP::Server::Async’s author) is already on the case.

“eleven” lands for panda!

FROGGS has merged a panda branch called “eleven”, which makes use of the CompUnitRepo functionality of rakudo. The effect is that installed distributions will not be stored as files on disk, but go into a database like thing. This is done to be able to install (and locate) several same-named versions of a distributions, or distributions made by different authors. These can now coexist, though as a consequence every distribution must have a “provides” section [^1] in its META.info.
Such a “provides” section (or an entire META.info) can be generated by running “panda gen-meta”. Modules without a proper “provides” have a panda icon with a strike-through “S11″ flag next to them on the modules list, so if you want to dip your toe into contributing to Perl 6, here’s a very simple way to do so… :)
The CompUnitRepo database is not just there to allow looking up namespaces by distribution version or author, it also copes with the issue that arises when two distributions try to provide a binary/script with the same name. The CompUnitRepo will install wrapper scripts instead, which take –auth, –name and –ver options to select a specific binary or script to execute. If the search result is ambiguous a list of candidates is displayed to allow to refine the search criteria.
Next on FROGGS’s TODO is to implement %?RESOURCE [^2] and also to look into any fallout that happened because of his latest work.

[^1] http://design.perl6.org/S22.html#provides
[^2] http://design.perl6.org/S22.html#resource

And just like that, another week has already come and gone. This week, I’ll actually have a look at finding some interesting snippets for you to post on a different day. I’m thinking Thursday would be a good day to try. Thanks for the warm words, everyone :)

See you on Thursday, or at least on Monday :)

2015.12: A release. Also, lazy deserialization makes a comeback

On Friday, the 20th we got a Rakudo release, followed swiftly by a Star release on the 21st and a Rakudo Star .msi installer this monday. There’s also one Rakudo Star Docker Image based on Fedora 21 by mj41 and one Rakudo Star Docker Image based on Ubuntu 14.04 by hoelzro. On top of that, FROGGS wrote down how to create the Rakudo Star .msi packages. Big thanks to FROGGS for the Rakudo release and the MSI, moritz for the Rakudo Star release and everyone else involved in the Release process and all the tasks surrounding it!

Performance

A long time ago, MoarVM learned how to delay deserialization of a whole bunch of objects until they are referred to, but there was at least one bug that was caused by it and so it was disabled for a long while. That feature was re-enabled just after the release, so that we get quite some testing by the community to find out if it still causes issues and this time hopefully get rid of them, since the lazy deserialization gives us much better memory usage and speed when starting up a program that doesn’t use every class there is.

That’s not the only change we can look forward to, though. Jnthn is currently working on improving the impact of Frames on the garbage collector a whole lot, which (in my expectation) should improve memory usage as well as collection times.

In addition, Jnthn will fix my attempts at teaching NQP’s code-gen and Rakudo’s optimizer about references to local variables, which will recover the performance hit we took when our support for native variables and attributes has been vastly improved recently.

There was also a commit that prevented $*VM from being initialized on every startup that dramatically decreased startup time, but that caused a problem with installing modules, so it was reverted. Maybe we’ll end up getting it back this month with a proper fix for the underlying problem!

There’s also two things yours truly is working on: Making calls to a repr’s operations (attribute access, pushing/popping, positional and associative accesses, …) less overhead-ridden in the JIT compiler and merging facts about registers in spesh when Blocks in the Command Flow Graph come together. They aren’t ready for merging yet and also don’t seem to cause significant performance improvements, but I’ll keep looking for ways to make things faster …

Platform things

Rakudo used to follow parrot’s install directory layout, which put things in $PREFIX/languages. We changed that to $PREFIX/share instead; you may have to re-install modules and run Configure.pl again for all parts of your Rakudo installation if you’re staying on the bleeding edge via a git checkout. If things get very wonky with rakudobrew, consider nuking the subdirectories of ~/.rakudobrew …

This change was made to make rakudo and nqp much more friendly towards linux distributions.

Also, the docker image efforts by hoelzro and mj41 will be combined into a more official rakudo star docker repository.

What’s also very worth mentioning is Coke’s ongoing work to get our stuff into the official MacPorts repository. There’s only MoarVM so far, but NQP and Rakudo shall follow.

Language Features, Syntax, Bug fixes

usev6 fixed the interaction of “last” and the sequence operator, making sure that code like this

perl6 -e 'say 10, 9, 8, { $_ - 1 || last } ... *'
10 9 8 7 6 5 4 3 2 1 1

now properly drops the last entry in the list.

Raydiak (and Mouq) implemented the simplest case of an angle-bracketed Rat literal, like <1/5>. In the future, there’ll also be support for lists of Rat literals like <1/2 1/3 1/4>. I may have to bug them to also make Complex literals work in angle brackets :)

Psch fixed a problem in Rakudo-JVM’s Binder that’d cause coercive types to pass the original value rather than the coerced value to a function’s body. After that, he also implemented nqp::getuniname on Rakudo-JVM which powers Perl6’s uniname sub. A few improvements for the Eval Server have also landed.

Thanks to TimToady the “cannot call $sub_or_method, none of these signatures match:” error message also shows the capture that was passed and caused the “no match” exception. The robustness of Mu’s .perl method has also been improved and its output prettified a tiny bit.

Lastly, Mouq fixed the combination of Bag and hyper operators, such as “Bag.new(…) »-» 1″.

Another thing happening in a non-merged branch right now is more features and improvements for supporting CUnion meant for (but not limited to) NativeCall by FROGGS.

Documentation

moritz, ptc and others have been improving the official doc.perl6.org contents again. That’s Way Cool if you ask me! As always: feedback is very welcome on the IRC channel or the perl6-users mailing list, or even in the comments section of this blog.

Finishing up

I feel like I’ve missed a few things, but I’ve been procrastinating this post far too long now and bedtime is coming up soon :)

Therefor I wish you a nice week and have fun trying out the newest Rakudo Star release!

P.S.: I’m thinking about adding a second series of posts to this blog. Once a week I’d look through all the snippets evaluated with our evalbot on the IRC channel and pick out one or two and briefly describe what I like about them. What do you think about that idea?

2015.11 Native arrays arrive

Friday the Thirteenth turned out to be a very fruitful day, as jnthn++ could finally merge a lot of the work that he has been doing the past weeks in the background regarding native arrays. Native integer arrays come in the following flavours: int8, uint8 (one byte per element), int16, uint16 (two bytes per element), int32, uint32 (four bytes per element), and int64, uint64 (eight bytes per element). On top of that there are two generic native integer array types: int and uint, which are aliases for either the 32bit or 64bit version (depending on your hardware and/or build options). So what does that mean for performance in e.g. filling a 10M element array with an integer value:

my Int @A; my int $i;
while $i < 10000000 { @A[$i] = 1; $i = $i + 1 }   # 11.6 CPU seconds

Using an int array is more than 11x faster:

my int @a; my int $i;
while $i < 10000000 { @a[$i] = 1; $i = $i + 1 }   #  1.0 CPU seconds

If we compare this to Perl 5, we can see that Perl 5 is slower in this particular benchmark:

use integer;
my @a; my $i = 0;
while ($i < 10000000) { $a[$i] = 1; $i = $i + 1 } #  1.5 CPU seconds

However, Perl 5 optimizes the for idiom much better than using while. So if we use the for idiom, we get:

use integer;
my @a;
for (0 .. 9999999) { $a[$_] = 1 }             # 1.0 CPU seconds

So on par with Perl 5. Still, it is very exciting to see Perl 6 execution speeds approaching Perl 5’s!

Apart from native integer arrays, we also now have native numeric (real) arrays in 2 flavours: num32 (4 bytes) and num64 (8 bytes), and a generic num type that aliases to either.

The native arrays implementation is still a bit rough on the edges, specifically with regards to Listy functions such as .pick, .roll etc. The reason why these do not work yet, is clear: there is still some discussion as to how to resolve the issue, as it will involve some internal code shuffling.

Why is it so hard?

moritz++ wrote a very interesting blog post about why it is so hard to write a compiler for Perl 6. There were quite some comments on Reddit.

TheDamian in Amsterdam

Damian Conway gave his Everyday Perl 6 presentation in Amsterdam last week (like he did in Oslo the week before that). Some photo impressions from the lovely ambiance of the Tuschinski Theater. To quote TheDamian:

Perl 6 is a computer scientist’s dream come true. Its many advanced features (junctions, multiple dispatch, generics, delegation, grammars, lazy evaluation, infinite lists, hyperoperators, first class functions, coroutines, partial function application, macros, metadata, etc., etc.) offer the elite überhacker entirely new levels of awesome cosmic power.

Unfortunately, for the majority of us, those incredible new features are mostly just mysterious, scary, and off-putting. But all of those new features are also entirely optional. You don’t have to start out in Perl 6 writing autoparallelized infinite superpositions of multimorphic higher-order functions. You don’t even have to statically type your variables or encapsulate your objects.

Using attributes in regexes

jnthn++ added a compile time error when using class attributes inside a regex. From the commit message:

Regexes are methods on Cursor (or some subclass of it), meaning that
an attribute access inside of them is relative to that. Rather than
failing silently or exploding noisily at runtime, catch the problem at
compile time. The typed exception’s message explains the issue and
offers a suggestion.

Development Allsorts

  • FROGGS++ fixed a long standing bug on the JVM backend involving our scoped subs in combination with precompilation. This now allows e.g. the URI module to be used on the JVM backend.
  • hoelzro++ found a problem when multiple readers read from the same Channel. lizmat++ fixed this (at least for now) and hoelzro added a test for it.
  • lizmat kebab-cased some functionality (and added deprecations for the old names) in an effort to provide more naming consistency: is hidden-from-backtrace, is hidden-from-USAGE, use MONKEY-TYPING, RAKUDO-NO-DEPRECATIONS, Supply.on-demand, Supply.schedule-on.
  • If you’re running spectests regularly, you will see quite an increase in number of tests (36600+ tests now): these are mostly new native array tests.

Winding Down

timotimo++ was not feeling well enough yet to do the Perl 6 Weekly, so yours truly did it this time. timotimo, get and feel better soon!

Please let me know of any errors, comments, suggestions, like always!

2015.10 FOSDEM Videos and Role Conflicts

Unfortunately, timotimo++ wasn’t feeling well enough to do the Perl 6 weekly this week. So yours truly will be stepping in again. Best wishes to timotimo: hoping he will feel better again soon!

FOSDEM Videos

The FOSDEM videos of the Perl DevRoom have appeared online. I’d like to highlight the Perl 6 ones in this context:

Role conflict resolution, wraps and claim

skids++ has written a drive-by summary/followup thoughts on “the role debate”, after colomon++ had a blog post about it. Food for thought, indeed. Comments welcome, of course!

TimToady on a roll

While fixing many grammar related buglets, TimToady++ also made it illegal to create operators with spaces in them. A sane choice, if you ask me. He also specced and implemented three more pseudo-packages: LEXICAL (Contextual symbols in my or any outer’s lexical scope), OUTERS (Symbols in any outer lexical scope) and CALLERS (Contextual symbols in any caller’s lexical scope). The use of $/ as Perl 5’s input line separator, is now better detected and frowned upon with a clearer error message.

Array/Hash Internals Shakeup

It was discussed many times over the past weeks. This Sunday, lizmat bit the bullet by renaming all of the Rakudo specific array / hash access internal methods, and providing deprecation logic for the old method names. What does that mean? As a user, it shouldn’t matter much. As a module and/or internals developer, you may need to change some calls to these internal methods. Please check the ChangeLog for more information.

Nativecall C++ Integration

FROGGS++ has been steadily working on the integration of C++ with NativeCall in a branch. It is getting close to a merge, so hopefully we’ll be able to chalk up “C++ integration” on the list of NativeCall features next week.

Development Allsorts

The following features / bug fixes were implemented in the past week:

  • PairMap (Hash with fixed set of keys on creation, but variable values)
  • .pairup() (Create a list of pairs from a List of scalars, pairs, lists and hashes)
  • UInt (sub)type and coercer for unsigned integer.
  • is DEPRECATED works on subs/methods even if they don’t have phasers.
  • MoarVM gc crash that was blocking work in Inline::Perl5 has been fixed.
  • Most, if not all Expected Array[Str] but got Array[Str] errors fixed.
  • my Role $a .= new does not error out anymore.

And of course, many, many other bug fixes and optimizations, both on the MoarVM as well as specifically on the JVM backend (Mouq++, avuserow++, nwc10++, donaldh++ on top of the usual suspects).

Winding down

Please let me know of any errors, comments, suggestions, like always!

2015.09: Highlights of the Week, and also native refs!

This week two independent people have worked on independent syntax highlighters: ashgti is working on a .tmbundle for perl6 so that we can have highlighted code on github again. at the same time, literal has made an incredible amount of improvements to the perl6 highlighter for vim. The performance has increased significantly, but the highlighting is also much more precise. The new highlighting code relies on a feature that’s been added in Vim 7.4, but if your vim is older, the makefile will strip the new features out for you. Many, many thanks go out to these two!

Bugs, bugs, bugs

I think I haven’t mentioned this for a long while, but usev6 has been continuously plowing through RT looking at bugs that need tests or have been fixed in the mean time and closing them. Of course, usev6 isn’t the only one, as Mouq did many bug fixes as well. Having a more reliable number in the “new” and “open” columns for the perl6 bug queue in RT is very helpful. So thanks!

JVM Interop improvements

psch made the marshalling used for calling into native java classes’ methods able to handle multi-dimensional arrays, so more stuff can be called now. Also, there’s a function that allows you to simply evaluate some Perl 6 code from java, so you can have some embedded Perl 6 code in a java program.

Native refs

jnthn has been busy porting the native reference support to the JVM, so that it could finally be merged yesterday. Not only is this a very important prerequisite for having native arrays work properly, but it also fixes a lot of annoying NYIs that you could encounter when using native variables and attributes for performance reasons, like using ++ or trying to assign to multiple slots in a Buf or having a native attribute “is rw” with an accessor automatically generated for you. In the future, our optimizer (both the static one in rakudo and the dynamic one in MoarVM) will learn to take advantage of these native references to hopefully generate much faster code!

Smaller bits

TimToady lets rakudo cope with “keywords” being used for custom stuff, like subs, methods and classes. Ever wanted to have a class called “class”? Well, I’m not saying you should, but if the need ever arises, now you can. Also, functions called “if”. Another place where this used to be troublesome were sub names like “not-foo” or “so-what”. These work now.

On top of that, “smart quotes” are now supported:

perl6 -e 'say “quoting” ~ ‘quoting’ ~ ‚quoting’ ~ „quoting”'
quotingquotingquotingquoting

The perl6 testers website has also seen some more work (by yours truly, actually!), which means that the list of recent reports is back and report pages now allow you to navigate to the corresponding distribution in the breadcrumb list. You can check out the “most recent reports” page here.

Also, japhb implemented IO::Path.resolve, so that we can handle paths with symlinks in them. Kudos!

There’s also a bit of sad news: The Perl Foundation was not accepted into GSoC this year. That’s especially sad as there were multiple interesting suggestions for projects to have that relate to Rakudo and MoarVM.

That’s all from me for this week, but of course we’re not giving up yet ;) I heard from pmichaud that this week is the one he allocated time for GLR work in!

2015.08 – A Release, a Star and still picking up more speed

Release

Last week lizmat did a Rakudo release and on Saturday moritz did a Rakudo Star release (grab it on rakudo.org or get hoelzro/rakudo-star off the docker registry). Since the release happened on Thursday, a few of the changes mentioned in this blogpost are already part of the latest release.

Speed Improvements

This week I did a little contribution to performance as well! I made Bool.pick() (and Bool.roll()) 2x as fast and stumbled upon a problem in the compiler’s inlining decision-making that I’ve been chasing down over the last few evenings… But lizmat did a lot more than me, just like last week: Str.trans with single letter strings is now 2x as fast and Str.subst(Str, Str, :global) is 12x faster! There’s an additional gain of between 10% and 30% if you use Str.trans for replacing a number of characters with a single character. There’s also a tiny startup time improvement, but I didn’t measure it yet.

NativeCall improvements

FROGGS landed a branch that adds a “nativesizeof” op that will give you the size of … well, things :)

Later this week we’ll probably see support for unions (finally!) and a proper Pointer[…] role that’ll allow control over inlined vs non-inlined structs and unions in other structs.

Another branch that’s in the works, but will not be ready for merge this week, is “cpp”, which allows you to bind C++ classes. Currently only the name mangling scheme of gcc is supported, but it probably works with clang, too. Later on, more support will probably land.

Documentation changes

Many people have – rightfully – complained that documentation is still kind of weak for Perl 6. Thankfully, ptc and moritz have been busy (and not just this week)! Check out the latest state of the documentation or look at the recent changes. If there’s anything wrong or missing, feel free to drop by on IRC and report, or open an issue on github.

Miscellaneous fixes and changes

brrt fixed a bug where MoarVM’s file copy routine would stop copying data after 2gb.

Cool stuff in the ecosystem

There’s been a module called “Inline::Perl5” for a while now. It allows you to import and use modules written in perl5 in your Perl 6 program. There’s also “Inline::Python” which is basically the same thing but with Python instead of perl5. Both also allow you to derive Perl 6 classes from classes in the inlined language and then derive from Perl 6 classes in the inlined language.

Now there’s also “Inline::Perl6” – though I’m not sure if it allows the same subclassing thing I’ve described in the previous paragraph.

Those three modules were made by nine, but now there’s also “Inline::Lua” by raydiak.

It seems like Perl 6 is becoming quite useful as a glue language between many other languages. I wonder what Inline::$lang will be next! I heard rumors of Inline::AWK :)

There’s also a new interesting Slang out there now. It’s called Slang::Piersing and allows identifiers (like method names) to end in ! or ? (Technically that was the week before, but I neglected to cover it back then).

Miscellaneous

moritz added a new command to our lovely irc eval bot camelia that automatically uploads a profile of the snippet to the web. Here’s moritz’ blog post about this feature.

Ovid is praising subset types and gradual typing as found in Perl 6 in his blog post titled “Avoid a Common Software Bug By Using Perl 6“.

David Golden shares his “Thoughts on getting Perl 6 for Christmas” on his blog. Both positive and negative. He’s got a few good points as well as worries. I have a few notes on that post:

  • We are aware of the up-to-dateness problem of documentation – that it’s hard to see whether or not something is still relevant or correct. The “Perl 5 to Perl 6″ guide on moritz’ website (as linked to in the first paragraph) does have old dates in it, but it has been checked for correctness as recently as last September.
  • The “composable concurrency” talk is still relevant, though things have changed names (“Supply” instead of “Observable”, “start” instead of “async”, “vow” instead of “keeper”, …)
  • A brief survey of the Perl 6 IRC channel shows that nobody seems to know what “Synchronization” was even meant to refer to in the feature matrix. Given that we’ve got Locks and Semaphores and stuff implemented, I’m also not sure why it has a +/- entry.

All right, that’s it from me for this week. Have a pleasant Last Week Of February, everyone!

2015.07 – I like trains! Fast trains, that is …

This week I’m taking back over from lizmat who thankfully stepped in when other responsibilities kept me away from Perl 6 for a while. I kind of like her format a bit better than my big list of bullet points; I’ll try to do it more like her. Feedback is, of course, always welcome.

Performance improvements

After TimToady posted an implementation of the Textonyms task on RosettaCode that was too slow for comfort, lizmat had a look at the .trans method (which also powers the tr/// operator) and quickly made the common case of having only a Pair of two strings at least 20x faster. Another case of trans – only removing characters instead of replacing characters with others – got between 10x and 200x faster (depending on whether you remove all characters (200x faster) or no characters (10x)). These changes made TimToady’s code run 36x faster, the run time went from 6 minutes down to 10 seconds. Yay!

Availability of “long” type

FROGGS made a type “long” available – especially useful for use with NativeCall – to use whatever the current machine thinks a “long” is. This work also includes the ability to specify sizes for other things. In a related change, Buf objects will now report the correct size in “.bytes” when the size of an individual element is not one byte.

NativeCall’s in Rakudo now

NativeCall has been moved into the same repository that holds Rakudo. The reason for that is that their versions usually have to track each other very closely and having both in the same repository makes things a lot less strenuous.

Stricter laxness

Starting a Perl 6 script with a literal or a bare “v6;” is no longer considered obvious enough for parsing a script in lax mode. Especially since code run on the commandline with perl6 -e are lax by default.

An Interview and also a Blog Post

Paul Krill of Infoworld interviewed Larry Wall last wednesday (comments on reddit). On sunday our very own masak wrote a blogpost on mazes.

Syntax highlighting on doc.perl6.org

Thanks to paul t cochrane ([ptc] on the IRC) we now have syntax highlighted code blocks on doc.perl6.org and avuserow has a branch that uses Inline::Python instead of shelling out for a great performance improvement.

Native references

jnthn has been working on a branch “native-ref” that’s just recently been merged in MoarVM, but currently isn’t being taken advantage of in nqp and rakudo. It will – once merged – allow native values to be lvalues properly for things like having ++ on native values and having rw-accessors on native attributes in classes and working with values in Buf objects and friends. As far as I can tell, the code making use of the changes in the branch will be merged after the 2015.02 release has been tagged.

That’s it from me for this week. I hope I didn’t forget too much; I kind of have to get back into the swing of things :) As always, feel free to point out things that are missing or wrong. See you next week!

2015.06 In the wake of FOSDEM

Although this Perl 6 Weekly post only covers about 5 days worth of events, it was pretty busy. And of course I missed a few things in the last report, so I’ll mention them this week.

Interview with TimToady

Miyagawa++ held an interview with TimToady after his presentation at FOSDEM. The entire podcast can be downloaded, the transcript is partly paywalled.

Don’t you just love Rats?

CurtisOvidPoe expressed his love for Perl 6 and COBOL at just about the same time last week’s Perl 6 Weekly was published.

Sprouting Up in Brussels

Although not strictly about Perl 6, mdk++ published a very nice impression about his first visit to FOSDEM. And there are also pictures.

All Perl 6 Modules in a Box

moritz++ has created an experimental perl6-all-modules repository, which he mentions in a recent blog post. He used it to find out how many modules in the ecosystem were using Rakudo/NQP specific nqp::xxx() calls. It turned out to be about 8% of the modules: these will need a use nqp; command in the future.

Stats / Directions for doc.perl6.org

moritz++ has been very busy indeed. Apart from his tireless work in the background for doc.perl6.org (among many other things), he has also blogged about it. Of course, help in documenting Perl 6 stays welcome as ever! In this, I should also mention [ptc]++, for being an excellent editor (in the newspaper sense).

Rakudo Star 2015.01

Did I mention that moritz++ has been very busy? He released yet another Rakudo Star! And mj41++ made a Docker image!

Rakudo todo list

colomon++ made his own todo list for Perl 6.0. I can’t help but agree that currently seen precompilation issues, as well as instability when stressing parallel execution, are high on my list as well.

Parameter Coercers work!

jnthn++ has been very busy behind the scenes in Moar and nqp, in the “6pe” (for parametric extensions) branch. As jnthn explains:

6pe in Moar/NQP is a very general mechanism for type parameterization and (once I get to it) interning them across compilation units. The primary motivation was typed data structures, but it turns out the exact same mechanism is also a great way to implement coercion types and cache mixins. The reason I did coercion types and the mixin cache first is ‘cus (a) they’re useful, and (b) they’re a gentler first test for the 6pe implementation.

This branch was merged by moritz++, which made things like
     sub a(Str(Any) $foo) { }
(take Any, but coerce to Str) work, and make mixins such as:
     C.new does R
about 10x as fast, and make the core setting about 700KB smaller through re-use. More goodies will come from this work soon, but I’m going to defer telling you about them until next week.

Movement on the JavaScript backend for nqp front

hoelzro++ took all of the work that pmurias has done on the JS backend into a branch in the nqp repository. A big step forward into getting a truly functional JS backend for nqp, and in turn for Rakudo Perl 6!

Development Allsorts

psch++ fixed a problem with [R~], found by avuserow++. FROGGS++ fixed some issues with nativecast on the JVM. FROGGS++ also fixed some warnings regarding anonymous variables. Mouq++ improved the = meta-op, for cases such as $a >== $b, and enabled lexical term definitions like
     my \term:<∞> = Inf
lizmat implemented the nqp::readlink function on MoarVM/NQP with a lot of help from jnthn++ and FROGGS++. For now, it is only used in the newio branch.

Signing off for this week

As always, please let me know of any errors, omissions, additions!