Saturday, July 30, 2011

M0 Roadmap Goals for Q3 2011

M0 has been coming down the pipeline for several months. It's still pretty raw and has a number of known functionality holes, but it's getting better by the week. I'd like to make the next few stages of M0 part of our official roadmap, so this post spells out the overall plan and what I think we can accomplish in the next three months.

M0 currently exists as a fairly hacky Perl 5 prototype. This is of necessity because Perl isn't generally intended to operate at the level that M0 requires. Perl is still serviceable as a prototype implementation language, but the form that will be integrated into Parrot will be written in C. There will be many stages between now and when the M0 migration is complete, but the goal I'll focus on is noop integration. I'll explain what I mean by that below.

I see Parrot's migration to M0 falling into 7 stages:

M0 Prototype

We're working out bugs in the Perl 5 M0 interpreter and making certain that M0 will be a sufficient foundation for Parrot. M0 may change significantly but we're making an effort to stabilize it.

C89 Implementation

We're happy with M0 and have a reasonably efficient compiler-agnostic implementation of M0, written in C89, which passes all tests. Separate compiler-specific implementations are fine, but not a priority.

Noop Integration

C/M0 is linked into libparrot and exposes an interface that C code can use to call into M0 code. At this point no subsystems have been reimplemented in M0.

Mole

We specify and implement Mole, which will be a C-family langauge that compiles directly to M0. Writing M0 is painful (this was an explicit design goal), so Mole is what a large chunk of the M0 that implements Parrot will be written in. M0 bytecode is what will be run from Parrot, so other code generation possibilities exist.

Early Integration

We've started moving subsystems over to M0. The order of which systems hasn't been determined yet, but producing a complete list and making sure we're aware of the dependencies will prove important.

C/6model in Core

Having a solid implementation of 6model in core will eventually be a blocker. Implementing our current object semantics in M0, only to switch to 6model later isn't a wise use of our hackers' tuits.

Pervasive Integration

At this point, everyone can jump in. We have a couple major subsystems converted and have worked most of the kinks out of the process of translating C into M0. We'll be converting every subsystem that we can find to M0 and will have plenty of example code and documentation to lower the barrier to entry.

Complete Integration

Parrot has a fairly small core of C code consisting of little more than the M0 VM and the GC.


Committing to a timeline can be tricky. It's much more important to have an M0 that's thoroughly well thought-out than one that's usable by a certain date. That said, the M0 spec and prototype are coming along nicely. Completing the "Noop Integration" stage and possibly getting a solid Mole compiler by the 3.9 release are reasonable goals, depending on how many interested parties make themselves known. I'm happy to see that whiteknight has made C/6model one of his roadmap goals. C/6Model in Core is largely orthogonal to M0 except that it needs to be integrated and solid before we start translating Parrot's object-related C code into Mole.

Wednesday, July 20, 2011

When Interpreters Collide

Note: this post is about implementing an M0 interpreter in Perl and is more a lightly edited braindump than a polished presentation of a concept.

Recently some test failures in M0's test suite revealed that the prototype Perl interpreter had been sneaking some of its perl-nature into the implementation.  The M0 assembler had been storing all values as strings and the interpreter had been secretly using its perlishness to convert the number-like values into ints at runtime.  This doesn't work well for an M0 implementation because M0 needs to be very specific about the low-level behavior of an implementation and the way it treats registers.

Perl is not C, and the basic problem I'm running into is that Perl is not designed to operate at the low level that M0 (as it currently stands) requires.  M0 is all about bytes and assigning meaning to the value in a register by using a certain classes of ops on it.  Perl is much higher-level and doesn't even have a particularly strong distinction between strings and integer values.  If I want Perl to have string byte-oriented C-like semantics, it means that I'll be widely (ab)using the bytes pragma and pack/unpack.  This is doable, but it's also torturing Perl into implementing something even further from its intended use case than the current (and subtly-incorrect) M0 implementation already is.  sorear rightly freaked out when he looked at the M0 interp code, because it's doing something that Perl wasn't intended to do and something that Perl isn't particularly well-suited to.

Still, javascript has been used to emulate at least x86, 6502, Z80 and 5A22 and  with surprisingly reasonable performance.  Arguably that's also pretty far from javascript's intended use case, and still it works.  This many just be an issue of finding the least hacky way to do something inherently very hacky.

The alternative is to specify M0 to have flexible underlying semantics, but I don't know that it'd be either practical or advisable to go too far down this road.  It's worth giving some thought to making the M0 spec be minimally unnatural to implement in a high-level language, but M0 is by its nature a low-level beast.  Implementations are bound to reflect that to some
degree.

In the end, the best way forward will probably be to plow through the craziness of implementing a simplified CPU in Perl and look forward to building on chromatic's C implementation, where the intent of the implementation language is much closer to the aim of the project.

Sunday, July 3, 2011

Parrot Weekly News for July 3rd, 2011

Welcome to the first edition of PWN.  At YAPC::NA, long-time developer chromatic expressed frustration at the fact that Parrot as a community hasn't been effective in communicating the knowledge of its members.  IRC, while great for immediate communication, doesn't lend itself to transparency for those who don't have time to hang out on #parrot 24/7 or to follow our irc logs.  My hope for this newsletter is to make Parrot's development more transparent, even for those with only have an hour or two per week to keep up with Parrot.  I also hope that this will serve as a common channel of communication for all Parrot developers in order to provide a basic understanding of what's been happening in Parrot and what's needed.

YAPC::NA

The past week contained YAPC::NA, a grassroots Perl conference organized by the Perl community for the Perl community.  There were three Parrot-related talks given by kid51, dukeleto and me, and one Perl 6 talk given by colomon.  There was also a well-attended Parrot/Perl6 BoF session on Tuesday and a hackathon on Thursday.  The hackathon was largely focused on coding and didn't generate significant directed discussion.

kid51's 10 Questions

kid51 had a short talk in which he raised a number of important questions about OSS projects in general.  He then proceeded to apply those questions to Parrot, with less than stellar results.  He had some of good points, particularly that Parrot needs to become production-ready before it can be considered a true success, that Parrot needs to have a better-defined purpose and focus, and that the project needs to "get to the point".  Asking tough questions isn't usually fun, but kid51 did Parrot a great service by honestly and directly pointing out some of the flaws of our community.  I hope his feedback will lead to positive changes in the way we look at ourselves and the products we're producing.

kid51's slides and a recording of his talk are here.

dukeleto's Visual Introduction to Parrot

dukeleto presented an introduction to the world of Parrot.  His intent was to give Parrot newbies a high-level overview of Parrot, its community and its ecosystem.  It was lighter in content due to being targeted toward less experienced audiences.  Nevertheless, it was an entertaining talk for people who already knew Parrot and provided a novel metaphor for understanding VTABLEs.  Once we're based on 6model, I look forward to seeing what kind of metaphor he comes up with.

dukeleto's slides are here.

cotto's State of Parrot

I presented a talk on the state of Parrot just after dukeleto's talk.  I covered developments in Parrot over the past year, some of the issues we need to deal with and what we expect the future to hold.  The short version is that there are a number of problems that are keeping Parrot from realizing its potential, but I think we have it within ourselves to overcome them and to produce an exciting production-ready virtual machine with some novel and useful properties.

My slides are here.

colomon's Numerics in Perl 6

colomon gave a worthwhile talk about performing numerical calculations in Perl6, both in Rakudo and Niecza (pronounced "niecha").  The talk was a good display of how people are using code that's built on top of Parrot and Rakudo.  As with all beta software, there were places where colomon ran into holes in the implementations of both Niecza and Rakudo, but the talk was hopeful and make me proud to be a Parrot hacker.

His slides are here.

Parrot/Perl6 BoF

The Perl6 and Parrot BoF session was considerably more organization-focused than most attendees were expecting.  Although the majority of attendees were from Parrot, Perl 6 (Larry Wall) and Rakudo (colomon) were also represented.  A primary point was that Parrot need to get better at communicating communal knowledge among its members and users.

Someone also suggested an intriguing way of reframing participation in Parrot.  Many of us developers work to scratch our own itches, but question "What would you be doing if the Parrot Foundation were paying you a salary?" provided a new way to look at how we manage Parrot and spawned a couple threads on parrot-dev.  For my part, this question provided the morivation for putting together this newsletter.  I hope it will also provide a motivation for all developers to take a more complete view of Parrot.

Room For Improvement

In this section of the newsletter, I will highlight areas of Parrot that are ripe for optimization.  Due to YAPC::NA this newsletter is already filling up quickly, so I'll highlight just one area.

config_lib.pir creates a hash that contains all data picked up by Configure.pl during configuration.  It has more than 250 entries, the majority of which don't provide any useful information.  Figuring out which entries in the hash are necessary and removing all the rest will help trim Parrot's startup time and make parrot_config a bit easier to sort through.  If you're interested in this, drop by #parrot or parrot-dev and chances are good that someone will be able to put you to work.

Other possible areas for optimzation are listed on the following pages on our wiki.
http://trac.parrot.org/parrot/wiki/PerformanceImprovements
http://trac.parrot.org/parrot/wiki/chromaticTasks
http://trac.parrot.org/parrot/wiki/PCCPerformanceImprovements

Submitting

If you see an interesting conversation on either #parrot, parrot-dev or #perl6, please mark it by saying "PWN".  When preparing this newsletter, I'll search through irclog (moritz++) for any mentions of "PWN" and a summary of the conversation to the next edition of PWN.