错误：间接修改重载属性

I have an array that has values that look like:

{
  "1": {
    "id": "1",
    "user_id": "9",
    "post_id": "61",
    "parent_id": "0",
    "message": "Level 0.",
    "created_at": "2015-06-27 12:46:18",
    "updated_at": "-0001-11-30 00:00:00",
    "children": []
  },
  "2": {
    "id": "2",
    "user_id": "9",
    "post_id": "61",
    "parent_id": "1",
    "message": "Level 1.",
    "created_at": "2015-06-27 12:46:22",
    "updated_at": "-0001-11-30 00:00:00",
    "children": []
  }
}

I then have a loop that looks like:

foreach ($tree as $key => &$value)
{
    if ($value->parent_id != 0)
    {
        $tree[$value->parent_id]->children[] = &$value;
    }
}

This, however, returns the following error:

Indirect modification of overloaded property Comment::$children has no effect

Why? And how can I fix this?