Responsible generative AI

Lanternina uses generative AI to propose activities for an adolescent, produce printed pages and interpret returned material. This preliminary assessment considers the effects on the people involved, the controls present and the questions still to examine, drawing on Microsoft's responsible AI guidance.

Interest in the service of learning

Lanternina aims to help refine cognitive skills through activities that offer something to learn. The experiences should be engaging, but their appeal serves learning: it is not an end in itself and must not be used to create another dependency.

The activities should require a balanced level of effort, enough to exercise observation, reasoning and decision-making, with difficulty and support suited to the participant. The adolescent should be able to test their own ideas and stop freely. Enjoyment and time spent do not establish a benefit: the contribution to cognitive skills remains to be assessed in actual use.

Activities and participation

Instructions can involve tools or materials whose safe use depends on conditions in the home. Parental approval and content screening address parts of this risk, with different scopes.

The assessment also considers reading, visual and motor demands that can prevent participation, and feedback that could introduce stereotypes, humiliation or pressure to continue. Reviewing the material can reveal some of these problems; observing an afternoon is needed to understand how the adolescent experiences it.

Mitigations in place

The device can retrieve only activities approved by the parent that have not begun. Withdrawing approval removes an activity from that list. This controls initial delivery; later pages and continuations receive no second parental review.

Activity and continuation text passes through Content Safety before it is returned. A refusal blocks the whole result. The detectors cover hate, self-harm, sexual content and violence. They do not verify the physical suitability of an action in the home.

Local language checks intercept specific formulations of blame, hurry and judgments about the person. Before display, a refused proposed string is replaced with text already written in the plan. These checks reduce exposure to the formulations they recognize; equivalent wording can escape them.

Readings and memory

Interpretations of returned material inform continuations and memory. Retained originals allow a reading to be checked against its source. One concern is whether a description of a sheet becomes an unsupported judgment about the adolescent's ability, intent or wellbeing and influences later interactions.

Household data

Portal access requires an active invitation-bound membership. An adolescent can access only their own submissions; the parent can access the family archive. Revocation closes subsequent portal requests. Hub requests are independently authorized against a server-side household key binding, so changing the household in a URL is refused.

Photographs can include people or private information unrelated to the activity. The parent photo routes restrict access to the authenticated household. The family can delete individual photographs or confirmed selections; deletion records prevent an upload retry from restoring the same photo. These controls concern the photo archive, rather than every record derived from an activity.

During development, photographs, returned material, readings and diagnostics may be retained and shown to the authenticated family. This allows the original material to be compared with its interpretation. Retention policy will be reviewed after development.

The 14-day expiry of the optional temporary reading archive does not apply to every stored image or derived record. Deleting a photograph does not undo an already completed activity or automatically remove its derived summaries. Incoming photographs are validated as images, but that validation is not content moderation; the reading path does not screen their content with inbound-image Content Safety.

What remains to assess

Automated tests cover approval and withdrawal, blocking a refused generation, cross-household photo access and deletion followed by an upload retry. The existing trials add functional and synthetic checks. Further assessment concerns detector coverage, physical suitability and participation during complete activities in the home.

Response during use also needs examination. A dangerous instruction already printed remains with the family after a software correction. Handling that situation includes identifying the affected material and informing the family. An exercised AI incident response procedure remains undocumented.

Sources

Microsoft Learn: identifying potential harms and Azure OpenAI responsible AI guidance. The documentation chapter contains the fuller assessment and references.

This site sets no cookies, collects no statistics, and loads nothing from anybody else's server.