2017.47 More TPCiA Videos

In the past weeks, more videos from the The Perl Conference in Amsterdam have become available. Rakudo Perl 6 related videos that have been added:

Kudos to all behind the scenes who made the recording, processing and uploading of these videos happen!

Blog Posts

Core Developments

  • Samantha McVey made collapsing of MoarVM strands (the parts of a concatenated string that hasn’t been turned into a single string yet) about 4x faster in most cases. She also fixed a bug with the utf8-c8 encoding that would combine a synthetic code point with other code points when they shouldn’t. Finally she fixed a bug in the Unicode properties database (uppercase letters not returning the Lu General_Category).
  • Jonathan Worthington fixed a race condition related to cancellations and an optimizer issue regarding improper inlining of blocks.
  • Bart Wiegmans fixed an off-by-one error that would cause try/catch to sometimes fail on jitted frames.
  • Elizabeth Mattijsen fixed the RAKUDO_EXCEPTIONS_HANDLER=JSON feature (which outputs execution errors in JSON format) for those exceptions that contain other exceptions. She also made sure that a Proxy will no longer leak out of a Lock.protect block. Finally she made sure that an execution error in an END will not stop any other END blocks from being run. Oh, and she made concatenation of Buf and Blob objects between 1.8x and 7x faster.
  • And many other fixes and improvements.

Meanwhile on Twitter

Meanwhile on StackOverflow

Meanwhile on FaceBook

Format::Lisp is on the ecosystem. Not *quite* fully featured, but it handles some of the recursive directives, and the test suite will tell what it can currently handle.

Meanwhile on perl6-users

Winding Down

We’re still waiting for the 2017.11 release of the Rakudo Compiler. There have been a few blockers that turned up at the last moment, which appear to have been vanquished, at least temporarily. But not in time for the deadline for this Perl 6 Weekly. So please check in again for next week’s Perl 6 Weekly for the continuing story!


2017.46 Spesh Explained

Jonathan Worthington completed his four part blog about the MoarVM Specializer Improvements he did in the past 3 months, supported by a grant from the Perl Foundation. The four parts are:

  • Gathering Data
    In which he explains the path towards the new spesh worker thread.
  • Optimization Planning
    About the introduction of the planner of specializations.
  • Optimizing Code
    About the different forms of specializations and the facts they are based upon.
  • Argument Guards
    About making sure that code execution will remain correct when any of the prerequisites for a given optimization change.

Required reading for anybody interested in the current and future inner workings of the Moar Virtual Machine!

Rakudo Star 2017.10 Released

Steve Mynott did all the hard work again and released Rakudo Star 2017.10, now available for download. Precompiled binaries for MacOS and Windows (64 bit) are also available.

Perl 6 Advent Calendar

Zoffix Znet reminds us that it is still possible to claim a slot in the Perl 6 Advent Calendar for this year. We’d like to hear from you! Your Advent Calendar Needs You! Just let us know!

No More Grants This Year

The Perl Foundation has run out of allocable funds for grants for this year. Your donation or a donation by your employer will allow for more grant work to be supported next year. So please give kindly, especially if you need to finish off budgets before the end of the year!

Core developments

  • Daniel Green improved the performance of a very specific case of concatenation of strings from O(n²) to O(1). The reason: that specific case was used in a lot of benchmarks and the solution did not have any adverse effects on all other string concatenation.
  • Timo Paulssen fixed a segfault that would occur when trying to create a low level iterator on a type object. This bug was exposed by trying to do a .pick on an empty Bag or BagHash, which didn’t check for emptiness. That particular bug was fixed by Elizabeth Mattijsen. Who also made sure that safe signals are now handled by the timer worker pool, which should make it less likely that your control-c will be lost in a barrage of general worker tasks.
  • Samantha McVey improved error reporting on situations in quoting, s/// and tr/// , when a starter would be followed by a combining code point. She also revived a two-year old nqp-branch which brings us + support for %b in sprintf (which made a lot of TODO tests pass!).
  • Christian Bartolomäus implemented nqp::getport on the JVM backend (which made a lot of IO::Socket::INET tests pass on that backend) as well as generally making sure none of the MoarVM backend related work busts building the JVM backend.
  • Jonathan Worthington fixed a scoping issue on the CLOSE phaser.
  • Elizabeth Mattijsen also made sure that OS thread exhaustion is more gracefully handled by $*SCHEDULER: if there are no more threads available, it will now only try once a second to create another OS thread (instead of trying this 100 times / second, which would leave less CPU for the other threads to complete their tasks).
  • Nick Logan did some fixes and cleanups in Rakudo’s make testing.
  • Finally, Elizabeth Mattijsen continued to be carried away by work on Telemetry. Most important additions were an ad-hoc instrument that allows you to snap the state of a number of variables (Telemetry::Instrument::AdHoc), and an instrument that takes basic Thread usage data such as number of OS threads started / completed / aborted (Telemetry::Instrument::Thread).
  • And many other fixes and improvements.

Other Blog Posts

Meanwhile on Twitter

Meanwhile on StackOverflow

Winding Down

It was a bit of a quiet week: nothing on perl6-users or on perlmonks. Which is also nice every now and then, as it makes the work of yours truly for the Perl 6 Weekly easier. The coming weekend will see the Rakudo 2017.11 compiler release with already more than 300 commits under its belt. The coming weekend will also see yours truly giving a one-hour “Introduction to Rakudo Perl 6” presentation at T-Dose (in Dutch), as well as a Perl stand with a lot of Rakudo Perl 6 books and goodies. Whether or not you will be able to attend, it seems wise to check in again next week for more Perl 6 related news 🙂

2017.45 Uplink Established

Elizabeth Mattijsen got carried away trying to devise an API for getrusage. The result is a new set of modules available with use Telemetry. And a helper module for easy activation: snapper. The simplest use case: perl6 -Msnapper yourscript.pl6. This will start up a separate thread that will take snapshots every 0.1 seconds. Once it is done, it will display a report much like this on STDERR:

Telemetry Report of Process #72908 (2017-11-06T21:26:26Z)
Number of Snapshots: 7
Supervisor thread ran the whole time
Initial Size:        67676 Kbytes
Total Time:           0.58 seconds
Total CPU Usage:      0.59 seconds

wallclock  util%  max-rss
   105476  13.28    17680
   105220  13.27       12
   105178  12.73     1024
   105149  12.85      152
   105141  12.42        4
    49525  12.98       12
--------- ------ --------
   575689  12.92    18884

wallclock  Number of microseconds elapsed
    util%  Percentage of CPU utilization (0..100%)
  max-rss  Maximum resident set size (in Kbytes)

Even this hot off the press, Wenzel P. P. Peppmeyer took this new development to actually create a distribution that runs an httpd daemon in your program, so that you can look at usage data of your (long running program) in your browser!

But that’s only the tip of the iceberg: there are also a number of ways to get at system information ad-hoc, such as:

$ perl6 -MTelemetry -e 'say T<wallclock cpu max-rss>'
(140878 264269 84756)

Or plan your snapshots and reporting more precisely:

while $working {
    LAST snap;
    # do a lot of work
say report(:csv)  # output in CSV format

Other features of Telemetry include a Telemetry::Instrument role for creating your own “instruments” for plugging into this framework (with two Instruments already supplied: Telemetry::Instrument::Usage (basically giving all the data of getrusage + wallclock in microseconds) and Telemetry::Instrument::ThreadPool (giving all the data pertaining to actions of the $*SCHEDULER, such as being able to see when new worker threads are started). Exciting times!

LPW Rakudo Perl 6 Presentations

Well, a program as such hasn’t been decided yet, but it looks like the following Rakudo Perl 6 related presentations will be given at the London Perl Workshop on 25 November 2017 at the University of Westminster:

Hope to see you there!

Optimizing Code

Jonathan Worthington published another part of his MoarVM Specializer Improvements series: Optimizing Code. A long read, but if you fancy understanding MoarVM capabilities better, it is highly recommended reading indeed!

Building a Single Page Application with Cro

This seem to have slipped through the cracks the past weeks. Jonathan Worthington describes how to set up a simple Single Page Application using Cro as the backend, and webpack, ES6, React and Redux as the frontend (no prior knowledge needed). It shows a SPA for a (food/beer) festival where people could leave their tips about what’s hot and what’s not, and being able to see them in real time. It supports:

  • Submitting new tips (a POST to the backend).
  • Having the latest tips appear live (delivered over a web socket).
  • Being able to agree to disagree with a tip (also a POST).
  • Being able to see a list of the tips sorted most agreeable to most disagreeable (obtained by a GET).

A must read if you are looking into building fully interactive web-applications, specifically using all of the asynchronous functionality that Rakudo Perl 6 offers today!

Other Core Developments

  • Although not strictly a core feature, but in the eyes of yours truly as important to Rakudo Perl 6 as Rails was to Ruby: a new release of Cro (version 0.7.1). New features are: cro web starting a web interface for stubbing, running, tracing & viewing logs from services in development, the router having include/delegate support for composing routes. A large number of issues was fixed, especially in HTTP/2.0 support.
  • Brian S. Julin‘s Pull Request to support the $.^name syntax has been finally merged. I guess it got a bit Warnocked 😦
  • Elizabeth Mattijsen fixed the multi sub foo(:@c is copy) syntax, which apparently was overlooked during the GLR. She also made certain combinations of attribute initializations throw an X::NYI (Not Yet Implemented) exception with a workaround suggestion, rather than silently not initializing.
  • Samantha McVey sped up joining of large strings (like more than 300 graphemes) up to 10x.
  • Daniel Green implemented an nqp op to quickly create an Int from another Int, but without any mixins. He also sped up various types of interpolation into regular expressions.
  • Timo Paulssen improved the “no such attribute” error on a failing nqp::getattr. This has helped yours truly already a lot in the past week!
  • And many more smaller fixes and improvements!.

Other Blog Posts

Meanwhile on Twitter

Meanwhile on StackOverflow

Meanwhile on perl6-users

Winding down

Not a small Perl 6 Weekly again this week. It’s almost become a dayjob! But a fun one! So keep all the new stuff coming so yours truly can report them again in the next issue of the Perl 6 Weekly. See you then!

2017.44 Nom Mastered

It has been a long time coming, but finally the default branch of Rakudo Perl 6 is no longer called nom (for New Object Model), but called master (which is what most git tools take as the default default). More than 6 years after it became the default branch. For a lot of people involved in the development of Rakudo Perl 6 it almost feels like a rite of passage. Zoffix Znet keeps us up to date on these developments.

Advent Calendar 2017

It’s almost that time of the year again, with just over a month to go: the Rakudo Perl 6 Advent Calendar. Zoffix Znet issued the Call for Authors. Have an idea for a blog post? Don’t wait too long with it: only 15 slots left at the moment of this writing!

Rakudo 2017.10 Released

After a long period of labor, compiler release Rakudo 2017.10 has seen the light after the last showstoppery issues were resolved. The third release that Aleks-Daniel Jakimenko-Aleksejev has done in a row! Not too long after that Claudio Ramirez made sure the world knows about the Linux packages he has created from that compiler release.

New Rakudo Star in the works

Steve Mynott tells us that there is now also an RC0 (Release Candidate #0) release of Rakudo Star 2017.10 (draft announcement). An important change with previous versions of Rakudo Star is that panda is no longer shipped because it has been deprecated in favour of zef.

Binary RC0 releases for MacOS (DMG) and 64-bit Windows (MSI) and a new experimental linux cross distro 64 bit build (using AppImage) are also available. The AppImage was built using scripts by Samantha McVey and should work on any recent (3 year old or newer) mainstream Linux distro by setting as executable and running (no installation needed).

Please download and test! Please give any feedback on the #perl6 IRC channel, or create an Issue. It’s hoped the final Rakudo Star 2017.10 release will take place around November 8th.

Squashaton Again!

This week sees the first Saturday of the month, so that means Squashaton time! This time the focus will be on the documentation again, aka the doc repository.

Books, Books, Books!

Andrew Shitov announced a new book “Using Perl 6” for immediate availability. On FaceBook he described it like this:

About a year ago, I decided to write a book about using Perl 6. Later, the plans changed and I published “Perl 6 at a Glance”, after which I wanted to write “Migrating to Perl 6” but instead wrote “Perl 6 Deep Dive” for Packt Publishing. Here and there, I was giving trainings on Perl 5, Python, and JavaScript, and was always suffering from finding a good list of easy tasks that a newcomer can use to train their skills in the language. Finally, I made up the list and solved it in Perl 6. This is the content of “Using Perl 6” — a book with solutions to 100 programming challenges, from simple to intermediate, with explanations of the Perl 6 constructions used. (…) The PDF version can be bought and downloaded already today.

Meanwhile, it became clear that Moritz Lenz has another book planned for late December: Parsing with Perl 6 Regexes and Grammars. I wonder how long it will take before this overview becomes too crowded! Who would have thought that a year ago?

Near Future Of Programming Languages

Stephen Diehl provided an overview last January. Guess which language is missing? Fortunately, some recent comments on Hacker News (this and that) picked up on that.

my %h is Set = ...;

Elizabeth Mattijsen made my %h is Set = <a b c> finally work (of course, also for the other QuantHash types: SetHash, Bag, BagHash, Mix and MixHash). This can be especially useful for the mutable QuantHash types, as you can now:

my %h is SetHash = (integers from some source);
%h .= grep: *.is-prime; # only keep the prime numbers in %h

And any other action that you would like to do “in place”.

Other Core Developments

Most of the tuits this week were spent on fixing the showstoppery issues that were blocking the release. Aleks-Daniel Jakimenko-Aleksejev provided an overview of ticket activity of the past two weeks that more or less reflects that. Other notable developments were:

  • Zoffix Znet and Elizabeth Mattijsen worked on various aspects of the new ThreadPoolScheduler, focusing on efficiency and stability.
  • Zoffix Znet solved an issue with multiple initializations of native arrays and an issue that prevented getc from not waiting for a keypress on MacOS.
  • Stefan Seifert continued his work on JITting the interface with NativeCall, making the interface between Rakudo Perl 6 and e.g. Inline::Perl5 much more efficient.
  • And many other smaller fixes and improvements.

Blog Posts

Meanwhile on Twitter

Meanwhile on StackOverflow

Meanwhile on perl6-users

Meanwhile on PerlMonks

Winding Down

As one of the distractions this week, I was looking at Code Golf, which has a lot of entries for Rakudo Perl 6. A good sight to see! Now, if I could only figure out how one can see the actual code samples. Perhaps next week. Join us then for another Perl 6 Weekly!

2017.43 Hyper lands, racing…

The past week saw reliable support for .hyper and .race arrive, as described in Jonathan Worthington‘s blog post earlier this year. Now some operations can be made about 3x faster with just a few keystrokes. To give an example, these are some ways of obtaining the prime numbers below 100000:

$ time perl6 -e 'my @primes = ^100000 .grep: *.is-prime'
real 0m26.428s
user 0m26.440s

Where real is the wallclock time that passed, and user is the amount of CPU-time that was used. Now, by adding a simple .hyper:

$ time perl6 -e 'my @primes = ^100000 .hyper.grep: *.is-prime'
real 0m9.288s
user 0m31.354s

We go from 26 to 9 seconds wallclock time, which is almost 3x as fast. And if you’re not interested in the order of the final result, you can use .race:

$ time perl6 -e 'my @primes = ^100000 .race.grep: *.is-prime'
real 0m8.335s
user 0m32.106s

Which is well over 3x as fast.

A more real-world example is the test-t.pl test of Text::CSV. The --hyper version typically runs for 1.14 seconds, whereas the serial (normal) version runs for 2.57 seconds (timings all made on the quadcore machine that yours truly is using, so YMMV).

To execute code that only needs to be run once in a batch, you can use a once block:

$ perl6 -e '^100 .hyper.map: { once { say "started with $_" } }'
started with 0
started with 64

Unfortunately, it is not yet possible to use phasers inside hypered/raced blocks, mainly because we still need to figure out when e.g. a LAST phaser should fire: at the end of a batch? Or after the final call of the final batch?

Casual Contributors Welcome!

Ann Barcomb describes a number of useful strategies for managing what she calls “episodic contributors” in her blog post: “How to manage casual contributors to open source projects. I think we can all learn from this, so recommended reading!

More Perl 6 Videos

The uploads page of TPCiA videos shows some more Rakudo Perl 6 related videos this week:

Also, Rakudo Perl 6 – A Primer by Simon Proctor (from the last London Perl Mongers Technical Meeting). Enjoy!

2017.10 Release Delayed

Some showstoppery issues with Rakudo are delaying the 2017.10 release. As this release will most likely be used for the next Rakudo Star release, and thus will be used longer and more extensively, it was considered better to wait until these issues are dealt with, at least temporarily. So stay tuned!

Other Core Developments

As promised last week, a roundup of other core developments of the past 2 weeks:

  • Several issues with quoting in is-run on Windows were fixed by Zoffix Znet, allowing make test to run cleanly again.
  • Samantha McVey made IO::Path.parent take an Int to indicate how many levels to go up (default: 1).
  • David Warring made indexing into NativeCall‘s TypedPointer (aka AT-POS) at least 4x as fast.
  • Daniel Green improved the speed with which variables are interpolated in regular expressions (aka the internal INTERPOLATE subroutine).
  • Zoffix Znet fixed IO::Pipe.close so that the Proc object is always returned, instead of only by the .close on the last open handle. He also fixed some issues with Promise.in, and Promise.at, and very low values / negative values / values in the past.
  • Timo Paulssen added Promise.kept and Promise.broken as a quick way to generate a kept or broken Promise without needing to call Promise.new.
  • Zoffix Znet also fixed precedence issues with the Unicode versions , , and of the ASCII operators <=, >= and !=.
  • Elizabeth Mattijsen fixed an issue caused by a mismatch in the layout of the BUILDPLAN between nqp and Perl 6. She also improved the speed of initialization of native arrays and fixed issues with the handling of Failures by some set operators.
  • And many, many, many more smaller fixes and improvements, both internally as externally visible. Interested in seeing more? Check the ChangeLog or the latest commits.

Blog Posts

Meanwhile on Twitter

Meanwhile on StackOverflow

Meanwhile on perl6-users

Meanwhile on PerlMonks

Winding Down

Alas, no recent module information on CPAN just yet. We’ve been all too busy fixing bugs before the 2017.10 release. But you should check out the new MetaCPAN-like search functionality of modules.perl6.org! And while you all do that, I’ll be preparing for next week’s Perl 6 Weekly. See you then!

2017.42 Taking Ticketing Seriously

Aleks-Daniel Jakimenko-Aleksejev has been ticketing away! The result can be seen in an overview of status changes in RT tickets of the past week or so. He explains it like this:

The high number of [REGRESSION] tickets is caused by the concentrated effort to detect any unintentional changes in Rakudo Perl 6. I extracted all messages to the evalbot (e.g. “m:” on the #perl6 channel) from the last few years, and passed these code snippets to different versions of Rakudo Perl 6. These snippets were filtered by comparing the output and exit code, which resulted in a list of thousands of samples that produce different results on different versions. Most of these differences are due to bug fixes or other harmless reasons (e.g. current time being shown in the output), but there were some notable regressions. All regressions noticed during the manual inspection are now filed, and some of them are already resolved (with tests, of course). This is the second time we see dedicated effort to find otherwise hard-to-notice regressions, first time being in December 2016.

So it should be easier to keep the sanity of Rakudo Perl 6 in check!

One more TPCiA Rakudo Perl 6 Video

Samantha McVey‘s High End Unicode in Rakudo Perl 6 has arrived! Of course, you can still keep checking for more uploads in the coming weeks.

Blog Posts

Meanwhile on Twitter

  • Please welcome, CPAN6 by builtinperl.
  • Rakudo Perl 6 BuildPack by brian d foy.
  • 20% shaved off by Zoffix Znet.
  • Debugging an Amazon issue by Moritz Lenz.
  • Whole day Rakudo Perl 6 Tutorial at TPCiG by Jeff Goff.
  • !perl6 at DuckDuckGo by Zoffix Znet.
  • int32.new shouldn’t give you an Int by Zoffix Znet.
  • Meanwhile on StackOverflow

    Meanwhile on perl6-users

    Meanwhile on PerlMonks

    Lately, there have been some Rakudo Perl 6 related Meditations on PerlMonks. Yours truly feels they should be mentioned in the Perl 6 Weekly as well, although some troll hugging may be needed:

    Ecosystem Additions

    This week will probably be the last time you will see the ecosystem additions mentioned in the Perl 6 Weekly. Thanks to Zoffix Znet and Aleks-Daniel Jakimenko-Aleksejev we will soon have a page on that will show the recent uploads to CPAN (the new preferred way putting your module out into the wild). So here goes for the last time:

    Winding Down

    Being too tired to investigate the latest Core Developments of the past week, I will keep them from you until next week. Which will also coincide with the 2017.10 release of Rakudo Perl 6! So please check in again next week!

    2017.41 The Case for Empathy

    Yours truly is glad she can finally link to Ruth Holloway‘s impressive The Case for Empathy keynote at The Perl Conference in Amsterdam. Please excuse the poor sound quality for the first 7 minutes or so: kudos to Amanda who managed to recover and process the sound from that period and supply the subtitles.

    Other videos of TPCiA related to Rakudo Perl 6 are:

    It is expected that more videos will be uploaded in the coming week, so please check again from time to time!

    CPAN6 is here

    Zoffix Znet did a lot of work on getting Rakudo Perl 6 modules out of CPAN, just as he promised last week. He explains it all in his blog post CPAN6 is here (Reddit comments). So now we finally have a front-end to browse the available Rakudo Perl 6 distributions that have been uploaded to CPAN!

    The Second Bug Squashathon

    Last weekend saw the second Community Bug Squashathon, this time focused on fixing problems in the core of Rakudo Perl 6. And the results are in! Looking forward to the next Squashathon, on 4 November 2017!

    Inside the head of Larry Wall

    Joe Armstrong conducted a Q&A with Larry Wall at the Erlang & Elixir Factory SF Bay Area 2017 conference (Hacker News comments). Also recommended as a very nice intro to Rakudo Perl 6 and the thoughts that have gone into the 100 year language.

    Rakudo Perl 6 performance analysis tooling

    Timo Paulssen‘s grant proposal for Rakudo Perl 6 performance analysis tooling has been approved by the TPF Grant Committee. Yours truly can hardly wait to see the results of this grant come to fruition!

    Call for Rakudo Perl 6 Papers!

    This is the time to propose your Rakudo Perl 6 presentations! Please select one or more conferences:

    Core Developments

    • The new JIT was merged into MoarVM, and consequently into Rakudo’s main branch. Some teething issues were already fixed by Jonathan Worthington. The new JIT hasn’t yet given us any large speed improvements, but now more people can work on making the JIT smarter and thereby making Rakudo Perl 6 faster! Yours truly is looking forward to be able to help here. Interested in working on this as well? Check out the documentation.
    • Zoffix Znet, with some help of Dagfinn Ilmari Mannsåker made EVAL/EVALFILE aware of Buf and the utf8-c8 encoding, as was originally stipulated in S29. He also worked on making .gist on several classes a lot faster, and re-enabled the :bin parameter on slurp.
    • Daniel Green made sure that enum values are now also included in errors suggesting identifiers when an identifier could not be found.
    • Larry Wall improved several parsing error messages, specifically on the likely location of the error, e.g. by catching missing expressions on statement modifiers earlier. Christian Bartolomäus and Aleks-Daniel Jakimenko-Aleksejev did similar work on improving the error message formerly known as “Malformed my”.
    • Zoffix Znet added a --repl-mode parameter to Rakudo Perl6 6, specifically meant to make testing the REPL easier. He also created new REPL tester logic and adapted the tests to use the new REPL tester logic.
    • Aleks-Daniel Jakimenko-Aleksejev also improved many error messages related to mismatched quotes.
    • Jonathan Stowe provided several fixes related to NativeCall and also provided tests for those fixes.
    • Elizabeth Mattijsen made object creation where attributes have compile-time default values (such as has $.foo = 42) about 40% faster. She also improved bare startup time by making code, that would normally run during startup, be pre-compiled in the setting.
    • Zoffix Znet, with some persistent help of Aleks-Daniel Jakimenko-Aleksejev, made array slices with a lazy infinite list (such as @a[3..*]) about 4x faster.
    • And many more smaller fixes and improvements.

    Other Blog Posts

    Meanwhile on Twitter

    Meanwhile on perl6-users

    Winding Down

    Quite an eventful week again, with the JIT being merged and the Squashathon going on. More than 100 commits in the rakudo repository alone. Please check in again next week for more Rakudo Perl 6 news!