2015.08 – A Release, a Star and still picking up more speed


Last week lizmat did a Rakudo release and on Saturday moritz did a Rakudo Star release (grab it on rakudo.org or get hoelzro/rakudo-star off the docker registry). Since the release happened on Thursday, a few of the changes mentioned in this blogpost are already part of the latest release.

Speed Improvements

This week I did a little contribution to performance as well! I made Bool.pick() (and Bool.roll()) 2x as fast and stumbled upon a problem in the compiler’s inlining decision-making that I’ve been chasing down over the last few evenings… But lizmat did a lot more than me, just like last week: Str.trans with single letter strings is now 2x as fast and Str.subst(Str, Str, :global) is 12x faster! There’s an additional gain of between 10% and 30% if you use Str.trans for replacing a number of characters with a single character. There’s also a tiny startup time improvement, but I didn’t measure it yet.

NativeCall improvements

FROGGS landed a branch that adds a “nativesizeof” op that will give you the size of … well, things :)

Later this week we’ll probably see support for unions (finally!) and a proper Pointer[…] role that’ll allow control over inlined vs non-inlined structs and unions in other structs.

Another branch that’s in the works, but will not be ready for merge this week, is “cpp”, which allows you to bind C++ classes. Currently only the name mangling scheme of gcc is supported, but it probably works with clang, too. Later on, more support will probably land.

Documentation changes

Many people have – rightfully – complained that documentation is still kind of weak for Perl 6. Thankfully, ptc and moritz have been busy (and not just this week)! Check out the latest state of the documentation or look at the recent changes. If there’s anything wrong or missing, feel free to drop by on IRC and report, or open an issue on github.

Miscellaneous fixes and changes

brrt fixed a bug where MoarVM’s file copy routine would stop copying data after 2gb.

Cool stuff in the ecosystem

There’s been a module called “Inline::Perl5” for a while now. It allows you to import and use modules written in perl5 in your Perl 6 program. There’s also “Inline::Python” which is basically the same thing but with Python instead of perl5. Both also allow you to derive Perl 6 classes from classes in the inlined language and then derive from Perl 6 classes in the inlined language.

Now there’s also “Inline::Perl6” – though I’m not sure if it allows the same subclassing thing I’ve described in the previous paragraph.

Those three modules were made by nine, but now there’s also “Inline::Lua” by raydiak.

It seems like Perl 6 is becoming quite useful as a glue language between many other languages. I wonder what Inline::$lang will be next! I heard rumors of Inline::AWK :)

There’s also a new interesting Slang out there now. It’s called Slang::Piersing and allows identifiers (like method names) to end in ! or ? (Technically that was the week before, but I neglected to cover it back then).


moritz added a new command to our lovely irc eval bot camelia that automatically uploads a profile of the snippet to the web. Here’s moritz’ blog post about this feature.

Ovid is praising subset types and gradual typing as found in Perl 6 in his blog post titled “Avoid a Common Software Bug By Using Perl 6“.

David Golden shares his “Thoughts on getting Perl 6 for Christmas” on his blog. Both positive and negative. He’s got a few good points as well as worries. I have a few notes on that post:

  • We are aware of the up-to-dateness problem of documentation – that it’s hard to see whether or not something is still relevant or correct. The “Perl 5 to Perl 6″ guide on moritz’ website (as linked to in the first paragraph) does have old dates in it, but it has been checked for correctness as recently as last September.
  • The “composable concurrency” talk is still relevant, though things have changed names (“Supply” instead of “Observable”, “start” instead of “async”, “vow” instead of “keeper”, …)
  • A brief survey of the Perl 6 IRC channel shows that nobody seems to know what “Synchronization” was even meant to refer to in the feature matrix. Given that we’ve got Locks and Semaphores and stuff implemented, I’m also not sure why it has a +/- entry.

All right, that’s it from me for this week. Have a pleasant Last Week Of February, everyone!

2015.07 – I like trains! Fast trains, that is …

This week I’m taking back over from lizmat who thankfully stepped in when other responsibilities kept me away from Perl 6 for a while. I kind of like her format a bit better than my big list of bullet points; I’ll try to do it more like her. Feedback is, of course, always welcome.

Performance improvements

After TimToady posted an implementation of the Textonyms task on RosettaCode that was too slow for comfort, lizmat had a look at the .trans method (which also powers the tr/// operator) and quickly made the common case of having only a Pair of two strings at least 20x faster. Another case of trans – only removing characters instead of replacing characters with others – got between 10x and 200x faster (depending on whether you remove all characters (200x faster) or no characters (10x)). These changes made TimToady’s code run 36x faster, the run time went from 6 minutes down to 10 seconds. Yay!

Availability of “long” type

FROGGS made a type “long” available – especially useful for use with NativeCall – to use whatever the current machine thinks a “long” is. This work also includes the ability to specify sizes for other things. In a related change, Buf objects will now report the correct size in “.bytes” when the size of an individual element is not one byte.

NativeCall’s in Rakudo now

NativeCall has been moved into the same repository that holds Rakudo. The reason for that is that their versions usually have to track each other very closely and having both in the same repository makes things a lot less strenuous.

Stricter laxness

Starting a Perl 6 script with a literal or a bare “v6;” is no longer considered obvious enough for parsing a script in lax mode. Especially since code run on the commandline with perl6 -e are lax by default.

An Interview and also a Blog Post

Paul Krill of Infoworld interviewed Larry Wall last wednesday (comments on reddit). On sunday our very own masak wrote a blogpost on mazes.

Syntax highlighting on doc.perl6.org

Thanks to paul t cochrane ([ptc] on the IRC) we now have syntax highlighted code blocks on doc.perl6.org and avuserow has a branch that uses Inline::Python instead of shelling out for a great performance improvement.

Native references

jnthn has been working on a branch “native-ref” that’s just recently been merged in MoarVM, but currently isn’t being taken advantage of in nqp and rakudo. It will – once merged – allow native values to be lvalues properly for things like having ++ on native values and having rw-accessors on native attributes in classes and working with values in Buf objects and friends. As far as I can tell, the code making use of the changes in the branch will be merged after the 2015.02 release has been tagged.

That’s it from me for this week. I hope I didn’t forget too much; I kind of have to get back into the swing of things :) As always, feel free to point out things that are missing or wrong. See you next week!

2015.06 In the wake of FOSDEM

Although this Perl 6 Weekly post only covers about 5 days worth of events, it was pretty busy. And of course I missed a few things in the last report, so I’ll mention them this week.

Interview with TimToady

Miyagawa++ held an interview with TimToady after his presentation at FOSDEM. The entire podcast can be downloaded, the transcript is partly paywalled.

Don’t you just love Rats?

CurtisOvidPoe expressed his love for Perl 6 and COBOL at just about the same time last week’s Perl 6 Weekly was published.

Sprouting Up in Brussels

Although not strictly about Perl 6, mdk++ published a very nice impression about his first visit to FOSDEM. And there are also pictures.

All Perl 6 Modules in a Box

moritz++ has created an experimental perl6-all-modules repository, which he mentions in a recent blog post. He used it to find out how many modules in the ecosystem were using Rakudo/NQP specific nqp::xxx() calls. It turned out to be about 8% of the modules: these will need a use nqp; command in the future.

Stats / Directions for doc.perl6.org

moritz++ has been very busy indeed. Apart from his tireless work in the background for doc.perl6.org (among many other things), he has also blogged about it. Of course, help in documenting Perl 6 stays welcome as ever! In this, I should also mention [ptc]++, for being an excellent editor (in the newspaper sense).

Rakudo Star 2015.01

Did I mention that moritz++ has been very busy? He released yet another Rakudo Star! And mj41++ made a Docker image!

Rakudo todo list

colomon++ made his own todo list for Perl 6.0. I can’t help but agree that currently seen precompilation issues, as well as instability when stressing parallel execution, are high on my list as well.

Parameter Coercers work!

jnthn++ has been very busy behind the scenes in Moar and nqp, in the “6pe” (for parametric extensions) branch. As jnthn explains:

6pe in Moar/NQP is a very general mechanism for type parameterization and (once I get to it) interning them across compilation units. The primary motivation was typed data structures, but it turns out the exact same mechanism is also a great way to implement coercion types and cache mixins. The reason I did coercion types and the mixin cache first is ‘cus (a) they’re useful, and (b) they’re a gentler first test for the 6pe implementation.

This branch was merged by moritz++, which made things like
     sub a(Str(Any) $foo) { }
(take Any, but coerce to Str) work, and make mixins such as:
     C.new does R
about 10x as fast, and make the core setting about 700KB smaller through re-use. More goodies will come from this work soon, but I’m going to defer telling you about them until next week.

Movement on the JavaScript backend for nqp front

hoelzro++ took all of the work that pmurias has done on the JS backend into a branch in the nqp repository. A big step forward into getting a truly functional JS backend for nqp, and in turn for Rakudo Perl 6!

Development Allsorts

psch++ fixed a problem with [R~], found by avuserow++. FROGGS++ fixed some issues with nativecast on the JVM. FROGGS++ also fixed some warnings regarding anonymous variables. Mouq++ improved the = meta-op, for cases such as $a >== $b, and enabled lexical term definitions like
     my \term:<∞> = Inf
lizmat implemented the nqp::readlink function on MoarVM/NQP with a lot of help from jnthn++ and FROGGS++. For now, it is only used in the newio branch.

Signing off for this week

As always, please let me know of any errors, omissions, additions!

2015.05 FOSDEM happened

The past week (well, actually 10 days by now), was mostly dominated by the FOSDEM happening in Brussels. The roster of Perl 6 related presentations was quite impressive:

Videos of all of the presentations in the Perl devroom, and TimToady’s presentation will become available in the coming weeks.

Production Ready Release This Year

TimToady++ announced that he expects that September 2015 will see a Release Candidate of Perl 6. And that barring any unforeseen circumstances of the “getting hit by a bus” type, we should see a Production Ready Release of Perl 6 before Christmas 2015. This got the intertubes buzzing. Before we all can rejoice, a lot of work will need to be done, specifically on the Great List Refactor (GLR), the Native, Shaped Arrays (NSA) and the Normalization Form Grapheme (NFG). And many other cleanups, as well as documentation patches and tests to be written. Your help is very welcome!

Public Service Announcements

  • pmichaud++ is preparing a blog post about the discussions within the Rakudo team while in Brussels. Be sure to follow his blog.
  • pmichaud++ also announced that in the future, non-core code will need to do a use nqp; to be able to access NQP functions.

Development Allsorts

You would think that not a lot happened in the past week because many contributors were working on their FOSDEM slides and/or travelling to Brussels. But it appears the reality is different. hoelzro++ worked on the sanity of type coercions. lizmat worked again on making adverbed slices faster (only 10% though, this time). Mouq++ allowed for indented code blocks in pod to contain empty lines (without breaking the code into separate entities). moritz++ moved the Real.sleep method to Numeric, and then DEPRECATED it. Mouq++ made auto-vivification on LoL’s work and implemented the semi-colon version of the MAIN sub. FROGGS++ and retopmuca++ allowed passing of Buf/Blob to NativeCall functions directly. And retopmuca++ and sergot++ got IO::Socket::SSL to wrap existing socket connections.

Signing off for this week

Sorry for this issue to be so late in the week. But FOSDEM messed enough with my daily rhythm as to need some time to recuperate. As always, please let me know of any errors, omissions, additions!

2015.04 Gotanda Lands

This week saw the landing of the Rakudo compiler release 2015.01, named Gotanda. Not much happened before the release was done by [Coke]++. Unfortunately, yours truly caused a problem in slices that would break some modules in the ecosystem, so within 24 hours, a hotfix release 2015.01.1 was made. This means that a new Rakudo Star release is likely: but as of yet nobody had the tuits to do this.

All of the exciting new developments happened after the release, so unless you’re living on the bleeding edge, you will have to wait for the next compiler release to see them.

Rakudo on MoarVM got 20% faster

Because jnthn bumped the optimization level on the MoarVM compiler (something that would previously break Rakudo), we now have a faster Rakudo on MoarVM. Typical startup time is 20% less, and the spectest runs 20% faster as well. And without breaking things as far as we know.

Much more documentation

Many people have been working on documentation, so it gets more and more worth it to check doc.perl6.org if you have a question or want to look up specific subjects. moritz++ for instance, has documented Proc::Async for the first time (other than what S17 already specified in a less readable manner).

Claiming a method when composing

CurtisOvidPoe++, while preparing for his presentation at FOSDEM, found that the current way conflicts are handled when composing roles into other roles, and roles into classes, may be confusing and present a “silently do the wrong thing” issue. In a discussion with, among others, TimToady and jnthn, a claim keyword was suggested. Which was promptly implemented by jnthn in a branch.

Programming in Perl 6 considered hazardous

CurtisOvidPoe was very busy in the past week. He also tweeted about it:

I strongly recommend *NOT* programming in Perl 6. Do it long enough and you won’t want to go back to Perl 5.

Now, if that isn’t good news, I don’t know what is :-)

Development Allsorts

moritz++ fixed a number of issues with Proc::Async and Supply.categorize. Yours truly made slices with adverbs, such as %h<a b c>:delete, up to twice as fast. atroxaper++ provided a –git-reference option to the configuration of Rakudo, as well as NQP, allowing you to specify local repositories, instead of fetching from Github all the time.

Already 25+ patches since this month’s release!

Signing off for this week

Please let me know of any errors, omissions, additions!

2015.03 Gathering Up Steam Again

It seems the dry spell of the Holiday Season has come to an end. Apart from the usual bug fixes and optimizations, and many new tests written or unfudged, the past week saw a few new features, some of them previously unspecced!

Quick way to make an Object (aka Typed) Hash

TimToady++ implemented a new construct:

As in: create a typed hash that can take Any object as a key without resorting to stringification (which is what the default hashes do). Compare:
say :{ ^10 => 42 }.keys[0].WHAT; # object hash
# (Range)
say { ^10 => 42 }.keys[0].WHAT;  # "normal" hash
# (Str)

This work was done as part of making the .classify and .categorize methods create Object Hashes, rather than normal hashes.

Include attributes in suggestion list

hoelzro++ provided a patch that will include names of attributes if an unknown variable is referenced.
class A {
    has $!foo;
    method a {
        $foo = 42;
===SORRY!=== Error while compiling -e
Variable '$foo' is not declared. Did you mean '$!foo'?

Work behind the scenes

jnthn++ has been busy again, but most of that work is the 6pe branches, so is not visible yet.

6pe = 6model parametric extensions. It’s a small extension to the 6model primitives (that all of our type and OO support is built using) that support parameterized types, including interning of them across compilation units; our inability to do so to date has led to various issues with typed arrays/hashes especially in a pre-compilation setting. The 6pe branches in MoarVM and NQP focus around providing the handful of new nqp::ops that expose this, along with the serialization improvements to go with it. Meanwhile, the 6pe-mop branch in Rakudo is about exploiting these new things to get nice Perl 6 improvements; so far, mixin interning and coercion types, soon typed data structures, and a little later native arrays.

One of the results of this work, is that the 2015.02 release of Rakudo will see coercion types, as in:
sub foo(Str(Any) $x) {
    say $x.WHAT
# (Str)

Furthermore, mixins will be 10x faster, the CORE settings will be 1MB smaller, and other memory savings.

How to create a Grammar in Perl 6

A very nice article was written by sillymoose++.

The for ^100 { } optimization is restored

Somewhere in the past 2 months, we lost the optimization that speeds up simple loops like:
for ^100 { ... }
This probably happened because of some preliminary work on the GLR. As part of other streamlining work, lizmat figured out what the problem was. After some breakage on the JVM was fixed, this change (which makes these loops about 2.5x as fast) is now functional on every backend again.

Development Allsorts

psch++ did a lot of work on the JVM. raydiak++ spotted that there was no method version of substr-rw. lizmat implemented that, and made it about 3 times as fast. Various other array slice related optimizations / code cleanup were also done by lizmat++. donaldh++ did some amazing work with annotations, and optimizing bytecodes for integer constants 0..5, resulting in a smaller CORE setting.

Signing off for this week

Please let me know of any errors, omissions, additions!

2015.02 The beginning of the Party has been delayed!

The year started so well!

Seriously, already a delay? Not really. FOSDEM organisation has decided to move TimToady‘s presentation to Sunday, February 1st at 13:00 in the Janson room. This is great news, as that means that the Perl DevRoom at FOSDEM (on Saturday, January 31st) will not interfere with TimToady‘s presentation. The Janson room has a capacity for 1400 people, so be early if you want to have a seat: FOSDEM organisation is pretty strict with regards to reaching capacity of a room!

Perl6 Maven

szabgab has started writing new submissions to the Perl6 Maven site, such as Timestamp and elapsed time and Current Working Directory. The site itself runs on Bailador, the Perl 6 version of Dancer. A prime example of dogfooding!

Pair.gist has changed

A discussion between CurtisOvidPoe and TimToady about the difference between .perl and .gist, led to jnthn implementing a specific Pair.gist method. The most directly noticeable difference would be that:
   say (a=>42)
would now show
   a => 42
rather than
   "a" => 42
This caused some test failures, and some ecosystem breakage (mostly related to now faulty tests), but everybody involved felt that it is much better this way.

Development Allsorts

Over 50 commits this week to nom, some worth mentioning here: TimToady fixed << a b c ‘x’ >>[0] (it now returns “a”). The private method dispatcher was not being optimised away when it could: jnthn unbusted that, making private method calls as fast again as normal method calls (in the same situation). Rat % Rat did not work as intended, colomon added an infix:<%> candidate to fix this. Setting up $*DISTRO was not quiet in some situations, itz quieted those. Exporting an Enum, now also exports its values, thanks to jnthn. Warnings for undefined values in regexes were made much more useful by FROGGS. A lot of work was done on JVM dispatchers and JVM interop by psch (for the JVM backend only, of course). ab5tract did a lot of work for properly handling Mixes: there’s still some debate whether the currently implemented behaviour is correct.

Macro’s. Shaken. Not Stirred

Last week, I had completely forgotten to mention 007, masak‘s laboratory for “exploring ASTs, macros, the compiler-runtime barrier, and program structure introspection”. If you’re interested in exploring this environment, please work together with masak so that it can be meaningful for Perl 6 sooner, rather than later!

Signing off for this week

Please let me know of any errors, omissions, additions!

2015.01 Get ready to Party!

2014 has come and gone. The advent posts have come and gone. And timotimo++ is so busy, that he hasn’t got time to do the Perl 6 Weekly. For the coming weeks, yours truly will be trying to do the job at least as well as timotimo and GlitchMr have done in the past. So here are the changes in and around Perl 6 since the beginning of this year (roughly).


Finally, the year in which Perl 6.0.0 will* be released, has come. Will it be YAPC::NA? Will it be OSCON? Will it YAPC::Asia? Will it be YAPC::EU? Or will it be Christmas after all? Who knows? To fuel the betting, TimToady will give a presentation at FOSDEM on January 31st to tell us more. As a teaser, there’s also an interview available.

As an appetizer, lizmat has added a paragraph to the deprecation message that is shown if a program executed deprecated code:

Please note that *ALL* deprecated features will be removed at the
release of Perl 6.0.0 (expected sometime in 2015).

Precomp issues on Moar

Several efforts have been plagued by problems in pre-compilation of modules (e.g. v5, Inline::Perl5, S11/S22 development). It would appear that jnthn++ has been able to fix them, so no holding back on these efforts now!

Easier JVM interop

psch has landed the shortname MMD for overloaded Java methods, and constructors as well. As in, using native Java classes doesn’t need knowledge of method descriptors anymore. All is described in the (updated) Advent post.

20% faster startup on JVM backend

donaldh provided a patch for the JVM backend that disables bytecode verification. As donaldh stated:

The very idea of the bytecode verifier is a bit odd. The JVM needs to be robust when executing bytecode anyway. The verifier is intended to protect the JVM from malicious bytecode that has been downloaded from untrusted sources but that seems naively optimistic.

Doing a spectest on the JVM has gone down from about 2500 seconds, to about 2000 seconds (only processing one test script at a time).

Documentation of async (S17) and Metamodel

moritz++ has been busy writing initial versions of documentation of async features such as Supply, Channel and Lock, as well as the recently implemented (by jnthn++) Metamodel classes, such as Metamodel::Primitives.

Adverbed slices got a bit faster

While generally doing good things and resolving tickets, usev6 found that adverbed slices, such as:

%h<a b c>:delete;

were internally using junctions (|) instead of using a short-circuiting or (||). This has now been fixed by usev6++.

<=> on lists should distribute to elements

TimToady fixed the <=> operator on lists so that the contents of the lists matters, instead of just the number of elements of the lists.

low-level concurrency tests

japhb has been busy writing low-level tests for concurrency ops and repr’s in nqp. Unfortunately, they cannot be run by default yet (hangs on nqp-m, and crashes on nqp-j). So, Work In Progress!

newio branch

The “newio” branch attempts to make I/O in Perl 6 more sane, without breaking the current API’s too much. A lot of work has been done on it, but it is not ready for prime time yet. Unfortunately, lizmat was not ready to describe her vision in an Advent post. But she started work on the S16 Synopsis, also in a “newio” branch. So have a look if you want to remain up-to-date on that. Comments welcome!

Many optimizations, tickets resolved, tests written, bugs fixed

Even though the holiday season prevailed in the past weeks, giving many badly needed time off, others have taken the opportunity to resolve, unbug, test and optimize (at this point, already 107 commits to nom( the main branch of rakudo) since the 2014.12 release). Thank you all for that!

Signing off for this week

Please let me know of any errors, omissions, additions: they will probably make it to the Perl 6 Weekly next week!

*most likely

2014.46 and 47: a release, optimizations, parametric types, panda reporter, …

Wow, it’s gotten pretty late in the day by now; I’ve kind of been digging some more into MoarVM’s profiler (specifically the allocation logging part) and trying to use the gained knowledge to improve performance problems highlighted by our benchmarks …

It occurs to me, that a list of changes is kind of like a montage. And every hero needs a montage! This one is covering two weeks:

  • jnthn started implementing parameterized types, which is an important step towards having Native Shaped Arrays. The branches p6e in MoarVM and NQP contain that work. You can find the overview document on github in the MVM repository.
  • jnthn also improved things on Windows: failing tests and a gigantic performance hit from checking if a debug flag is set in the environment too damn often.
  • also, jnthn fixed floor and ceil on MoarVM giving bogus results when they get big numbers.
  • TimToady has found four distinct ways to not properly implement the second tiebreaker for longest token matching (“longest literal string wins”). He’s now working on the fifth way.
  • baby-gnu made MoarVM more suited for distributions to package it (don’t install MAST::Ops in /lib/MAST, for example)
  • FROGGS got the “reporter” branch of panda merged into master, so you can now upload testing reports by setting the environment variable PANDA_SUBMIT_TESTREPORTS
  • Another thing FROGGS did was make lists of adverbs without commas between them work properly in many more situations.
  • I implemented an optimization that flattens METAOP_ASSIGN into the calling code, thus getting rid of an invocation and taking a closure. This makes things like += noticably faster. METAOP_REVERSE (the R metaop) and METAOP_NEGATE (like in !==) got a similar treatment.
  • I also made DIVIDE_NUMBERS (which is used in the majority of rational number arithmetic operations) a tiny bit better by making its contents completely inlined on all backends.
  • Thanks to the profiler and spesh log output, I found that &prefix:<!> and &prefix:<not> were being treated very poorly by our code-gen; micro-optimizing it made these operators faster.
  • Other optimizations I’ve built didn’t have such a big impact: getting rid of some superfluous “set” instructions in MoarVM (equivalent to the simplest case of “mov” on x86), turning “indirect” operations into “direct” operations if the indirectness is actually known at spesh time, directly using the source of a boxing operation instead of unboxing directly afterwards, …
  • Also, I let the spesh log dumper also output the name of a lexical variable that’s accessed.
  • lizmat fixed a gigantic amount of warnings for clang – now actual problems should stand out much more readily even on Mac OS X.
  • lizmat optimized first-index, last-index and grep-index a bunch faster.
  • with some helpful nudging by lizmat, TimToady made the errors you get for trying to use negative indices in postcircumfix:<[ ]>. If it’s clear at compile-time, you can get a helpful error as soon as that.
  • JimmyZ added jit compilation support for some more ops: cmp_i, isnanorinf and cmp_n.
  • Mouq worked towards making slangs more easily usable from Perl 6 code, among others by introducing the ~ twigil.
  • usev6 did a whole lot of triage on our bug tracker and created a whole bunch of tests for bugs in RT.
  • smls made our documentation a bunch more helpful by giving each class or role a short description in the types list and Mouq added documentation for @(), %() and the type smileys.
  • nine has been working a whole bunch on Inline::Python for our lovely rakudo.

And here’s a few new modules that were added to the ecosystem:

On top of all that, lizmat did the 2014.11 rakudo compiler release last week! Thanks a bunch! :)