BombStrike's blog » javascript

Spidermonkey gets fixed and gets ready to move from “beta” to “stable”

Christophe Robin — Sun, 07 Mar 2010 22:07:31 +0000

Hello, it’s been a while since I last wrote something on this blog. Today I come back to tell you that version 0.1.4 of spidermonkey has finally been released and provide fixes for 3 bugs that where posted on the PECL bug tracker, I’ll do a small explanation of each bug and how they were resolved bellow. I also took care of integrating some patches for spidermonkey 1.9, added the config.w32 from hoshino at labs dot cybozu dot co dot jp and changed a couple stuff there and there in the source code. I also added regression tests for the solved bugs and updated some of the previous ones to reflect the corresponding changes.

Fixed bugs

Bug 16866:

This bug was pretty basic, when an empty string was returned from Javascript, the current PHP thread would segfault. The reason was simple, when Spidermonkey hold a variable that contains an empty string, it doesn’t possess a char* that would point to this empty string to save memory, so when I tried to retrieve this non-existing value to create an empty string in PHP, it would just segfault. I just solved this by checking the variable length in JS before doing the conversion, quite trivial compared to some bugs to come. It also solved bug 16876.

Bug 16890:

That one was another beast, when an object created in javascript was converted to PHP, it would become NULL on the next call. Clearly there was something weird going on here, after some painful debug, I saw that the issue came from a small caching system I had made that had no sense at all and was actually useless. Here is what it was doing:

When the javascript class was converted, it created the object and store it in a Zend HashTable with the JSObject id as its index. It would then retrieve the object from this “cache” when trying to access the object again. The basic idea at the time was to optimize objects that where converted back and forth a lot of time by pulling them from the cache, and preventing cyclic references by providing the cached objects without having to traverse it ( otherwise it would result in an infinite loop ). While the goal was great, the execution had several flaw, the biggest one was that for an unknown reason all objects that where stored were returned as NULL by the HashTable. But even if it had worked it would have caused issue with objects that were converted, modified in Javascript afterward and converted again, as the stored object would have been returned and not the updated one.

To solve this cyclic reference issue, I totally changed the approach by first removing this HashTable idea and then creating a basic structure that looked like:

typedef struct _php_jsparent {
JSObject *obj;
zval *zobj;
struct _php_jsparent *parent;
} php_jsparent;

When I run my js to zval routine, I pass a new parameter php_jsparent *parent that is NULL by default, if i’m converting an object, I create this jsparent object, store the current JSObject being converted, the stdClass zval that is being created and our parent if theres one and call the converting method recursively on each children. Now when *parent is defined, before converting an object I do a quick traversal of the parent links and if I find a JSObject *obj that is the same as the object I’m converting I just return the stored zobj, effectively solving the cyclic referencing issue.

Sample code:

a = { b : null }
a.b = a
/* in this case, trying to convert the a object result as a cyclic reference
when trying to convert the properties, the new code solves this when
trying to convert b, it checks it's parent which is in this case a, and
see that a == b, returning then the parent zobj and solving the issue */

Bug 16983:

Another tricky one. When the magic __get() method was defined on a class it became impossible to call any method on it.

The issue with this one was the JS_PropertyGetterPHP() function, which is a C getter function that is called on a Javascript object even if a property exists, I would then call the Zend getter function on the object and return the value if found. The main issue was that I wasn’t taking care of properties already defined on the JS side, and always trying to retrieve the data from the PHP side. This issue was solved pretty fast, I decided to reproduce the same functionality as PHP, that means that when this getter is called it first try to retrieve data from JS, in the case of methods, if would retrieve the property I had defined beforehand when creating the object and return it to our context. If it effectively doesn’t find anything on javascript, if then fallback on the PHP item, calling __get() if necessary.

Feedback

Most blocking bugs have been solved with this release, feel free to post on the bug tracker any new issue you would encounter and I’ll make sure to take a look. If no bugs are found, then I’ll make sure to finish a few things there and there and will release version 1.0.0 as stable to allow more peoples to use it safely.

Future tasks

There are several thing I’d like to do, like better debugging support using the tracing API that is being added in spidermonkey since 1.8+. But first would be support for closures in both directions, that means having access to JS function directly from PHP (that one is really hard, it will takes some time of tingling with the PHP internals to make it works) and assigning closures in JS (should be a lot easier). I already added the test on the SVN, and I’ll try to tackle this one in the next months.
If you have any idea of feature that you’d like to see implemented, feel free to ask in the comments

Spidermonkey in PECL

Christophe Robin — Tue, 24 Feb 2009 17:59:04 +0000

Today I finally released spidermonkey on PECL, you can find it here: http://pecl.php.net/package/spidermonkey/

The whole thing should work fine, I’m already working on several optimisations thanks to jorendorff from Mozilla on IRC. The next thing is to provide better error report, compilation/decompilation of scripts and more default functions for a bunch of things like Iterators and the like.

The last thing I added is a bunch of function applied to streams, here is an exemple:

/* create context */
$ctx = new JSContext();
/* provide fopen/fclose functions */
$ctx->registerFunction('fopen');
$ctx->registerFunction('fclose');
/* for output */
$ctx->registerFunction('printf');

Then in Javascript:

/* open file */
fd = fopen('somefile.txt', 'r')
while (line = fd.getline()) {
printf("%s\n", line)
}
fclose(fd)

/* open file */
fd = fopen('somefile.txt', 'w')
/* here fd.SEEK_SET is the default value and is optional */
fd.seek(1024, fd.SEEK_SET)
fd.write("some text here :D")
fclose(fd)

Current status and API of spidermonkey

Christophe Robin — Sat, 21 Feb 2009 20:10:38 +0000

Spidermonkey had many features and changes done this week, so let’s look at what it can actually do, and what it can’t.

Features

Execute Javascript ( well, of course it does )
Register PHP functions to use in Javascript
Assign PHP values in Javascript
Register Classes in Javascript, allowing the user to instanciate them.

Just with those functionalities, you can already do a great deal of things with JS, and extend it really easily.

Conversion

When working with each others, Javascript convert is types to Zval ( PHP native type ) and vice-versa. Most types are successfully converted, but there are some exceptions:

Javascript arrays are converted to stdClass due to the fact that they are stored as Objects by Spidermonkey.
Javascript closures are not yet supported, but because PHP 5.3 support closures too, it should be done soon.
Javascript regexp are not supported either, because there are no equivalents on the PHP side. In the end they’ll be converted to string.

Streams resources will soon have a prototype added by default with functions like read(), getline(), write(), etc…
Objects extending the Iterator abstract will also have a forEach function defined.

API

Here is the API:

class JSContext {
/* evaluate javascript source code
*
* @param string $script A string containing the javascript source code
* @return mixed The last value in the global scope is returned to PHP
*/
public mixed evaluateScript(string $script);

/* register a function for use in javascript
*
* @param callback $callback A valid callback that will be called
* @param string $name The name under which the function will appear in Javascript,
* mandatory for closures and recommanded for array($obj, 'function')
*/
public void registerFunction(callback $callback [, string $name]):

/* register a class for use in javascript so that it can be instancied using "new"
*
* @param callback $class_name A valid class to export
* @param string $exported_name The name under which the class will appear
*/
public void registerClass(string $class_name [, string $exported_name])

/* register a variable for use in javascript
*
* @param string $name The name of the variable in Javascript
* @param mixed $value Value for the variable, objects and resources
* are passed by reference
*/
public void assign(string $name, mixed $value)

/* those function allow you to play with the version the engine is running
* it's not totally compatible and most versions will not be loaded by
* the engine, you can still have a try with the constants defined above */
public mixed setVersion(long $version)
public long getVersion()
public string getVersionString(long $version)
}

This is the current status at revision 38. Next revisions will be aimed at bug correction and error management. If you have any ideas for features, feel free to comments this post.

New feature for spidermonkey: registerClass()

Christophe Robin — Sat, 21 Feb 2009 00:51:28 +0000

As I explained in my previous blogpost, you can export objects to spidermonkey by using the assign() function on this object. While it’s enough for basic cases, It’s not enough for complex situations where Javascript need to create itself objects instances.

Solution

Tonight I worked on the solution named “registerClass()“. As examples are better than words, here is how it works:

/* first create context */
$ctx = new JSContext();
/* register mysqli */
$ctx->registerClass('mysqli');

Then in your javascript source:

db = new mysqli('host', 'user', 'pass', 'db')
res = db.query('SELECT * FROM t1');
while (line = res.fetch_assoc())
{
// do something with line
}

As you can see, this function allow you to instanciate objects yourself in Javascript, providing more power and flexibility. This function will be available tomorrow morning in the SVN.

Bringing Javascript to the server using PHP * update *

Christophe Robin — Thu, 19 Feb 2009 14:38:09 +0000

I’ve been working for the last month on my first PHP extension since 2 years ago, a nice way to learn the new stuff Zend had in store for PHP 5.3. This extension is named SpiderMonkey, and embed the original engine made by Mozilla and available on most Linux distributions.

SpiderMonkey

SpiderMonkey is the Javascript engine used by Mozilla’s main products. It provide a nice C API for executing and interacting with Javascript. I base myself on the 1.7.0 version, which is the latest stable version. 1.8.0 is supposed to come soon but is still under development.

Usage

The API is straight-forward, a Javascript Runtime is first created, it’s a container for all “global” variables and Javascript Contexts. Contexts is the global scope where your program runs, all scripts executed on the same contexts will share the same global variables.

This part changed in rev. 38, see below

/* Initialisation of extension */
$rt = new JSRuntime();
/* Create a single context, most application will only use
* one context while a server ( like the pinetd httpd server )
* would create a context by request */
$ctx = $rt->createContext();

Since rev. 38, you don’t need to instanciate the JSRuntime anymore

/* Create a single context, most application will only use
* one context while a server ( like the pinetd httpd server )
* would create a context by request */
$ctx = new JSContext();

Once the context created, you’ll want to do two things:

Assign values from PHP to JS
Register functions from PHP to JS

For now, variables assigned from PHP are converted to a new Javascript variable and changing them in JS will not affect the original PHP value, while it’s supposed to be allowed later ( using a different function that will assign the variable as a reference and not a copy ), it’s already enough to provide $_GET, $_POST, $_FILES and the likes to JS. The only case where this change is for objects and resources, which keeps pointers to themselves thus allowing you to export a $dom variable containing a DOMDocument and acting on it by the mean of the object methods ( the object properties are not available ).

Exported functions are declared in the global scope.

Here is an exemple with mysqli:

/* Create a new database object */
$mysqli = new mysqli("host", "user", "pass", "db");
/* Allow Javascript to call sprintf under the name sprintf */
$ctx->registerFunction('sprintf', 'sprintf');
/* Provide our db connexion to Javascript */
$ctx->assign('db', $mysqli);

Then in Javascript you’d do:

/* i know the sprintf is not the best way to do it, but for readability
* i'll stick with it in this sample */
res = db.query(sprintf('SELECT * FROM my_table WHERE field = %s', db.escape_string(dynamic_data)))
if (res) {
while (line = res.fetch_assoc())
{
printf("%s\n", line.field2);
}
}

Because this is based on PHP 5.3, you could also use closures in the assign function, allowing things like:

$ctx->registerFunction('foo', function() { echo "bar"; });

Allowing you to create a function foo() that echo “bar” when called.

Future

A lot of new features are already being implemented, here is a non-exhaustive list:

Allowing to pass all variables as reference, allowing the Javascript to modify the PHP value
Allow to register “classes” prototypes, like:
$ctx->registerClass('DOMDocument');

Then you could just do “var a = DOMDocument();” in Javascript to instanciate the object.
Autoregister a prototype on some types of resources. A Stream resource could then be assigned in JS, and you could call stream.write, stream.read, etc…

How-to test ?

The whole source code is available on a public read-only SVN: https://ookoo.org/svn/pecl-spidermonkey. Be careful, you need PHP 5.3.0 beta 1 to compile it ( it will not even compile on PHP 5.2 ). There may be some issues when dealing with many objects so don’t use it for production purpose.

svn co https://ookoo.org/svn/pecl-spidermonkey
cd pecl-spidermonkey
phpize
./configure
make
./test.sh