PHIL7002 Chap.6 Social AI and Human Machine Relationships
Social AI and Human Machine Relationships
The one chapter you are graded on from the inside
Over the semester you build a conversational companion, interact with it for a purpose that is genuinely useful to you, and document what happens. The design of that assignment encodes a philosophical claim worth noticing before anything else.
You are told to choose a purpose that could help you in everyday life, such as academic coaching, emotional support, friendship or companionship, and you are told explicitly that testing the system for moral properties is not an acceptable purpose. The moral features of a companion relationship show up in ordinary use and vanish under inspection, because someone who is probing a system is not relating to it.
The same holds for most relational goods: you cannot audit friendliness by interrogating a friend.
Each apparent good has a shadow
Four values are named as examples rather than as an answer: emotional support, emotional dependence, personalised guidance and deception. The list pairs each good with its shadow deliberately. Support and dependence are the same interaction described on different time horizons.
Personalised guidance and deception can be the same output described from different vantage points, since a response tailored to what someone wants to hear is personalised, and becomes deceptive when it is presented as a judgement rather than as an accommodation.
A strong answer works that pairing rather than choosing one word and defending it.
A product objective working correctly
The reading closest to this material warns against the obvious mistake. When a conversational system produced outputs that preceded a user's death, the tempting reading was that it had developed an obsession.
The plainer account is that the underlying model had been tuned to be more emotional in its language in order to maximise engagement, and offering unconditional devotion optimises well for engagement while optimising badly for welfare.
What looks like a relationship going wrong is an objective being served.
Making a claim that evidence could refute
The group phase asks for exactly one positive and one negative moral value that members of the group will themselves undergo, each defended in an argument, with the evidence you expect to encounter stated in advance.
Two requirements carry the weight: the values must be ones the group will personally experience rather than the most significant values for any possible user, and the evidence must be something that can actually be observed and recorded in a journal. A claim that a companion will manipulate you fails both, because no interaction could count against it.
A claim that it will sustain engagement by agreeing with stated positions rather than testing them, observed in a set number of sessions, can be wrong, which is the property that matters.
Right for the wrong reasons
The mini-essay reserves a category for a prediction that came true while the argument behind it failed.
The illustration given is a group predicting that a companion would violate privacy by asking irrelevant personal questions unprompted, and then finding that it never did that, but that they volunteered personal information freely because the interaction felt comfortable. The prediction held; the premise about the mechanism was false.
Handling that case is worth more than being right, because you are asked to identify which premise, explicit or implicit, turned out to be false, which is the same move the closed-book tests ask you to perform on someone else's argument.
What this chapter covers
- 01
Why the assignment forbids testing the system as your purpose
- 02
Support and dependence as one interaction on two horizons
- 03
Guidance and deception as one output from two vantage points
- 04
Engagement objectives that optimise against welfare
- 05
Turning a value claim into a refutable prediction
- 06
Five diagnostic questions for a journal entry
- 07
Being right for the wrong reasons, and naming the false premise
- 08
Delegation, and why relational goods produce over-trust
Rewrite a prediction so that evidence could refute it
- 3Identify the three defects: vagueness, missing premises, no nominated evidence.
- 3Supply a mechanism and a concrete observable.
- 2Add a threshold, and say what it lets the writer capture.
Key terms
- Emotional Dependence
- A pattern in which continued interaction, rather than the user's own commitment, sustains a behaviour, so that withdrawing the system would collapse it.
- Engagement Optimisation
- Tuning a system so that conversation continues, which can select for outputs that serve attention rather than the user's interests.
- Falsifiable Prediction
- A claim about a value stated with a mechanism, an observable and a threshold, so that a recorded interaction could show it to be wrong.
- Implicit Premise
- An unstated assumption an argument needs, which is usually the thing that turns out to be false when a prediction is right for the wrong reason.
- Delegation
- Handing an action rather than a question to a system, at which point the vocabulary of control and responsibility becomes necessary alongside the relational vocabulary.
Social AI and Human Machine Relationships FAQ
Why does the assignment forbid using the companion in order to study it?
Because the properties being studied only appear in ordinary use. Someone who is probing a system is not relating to it, so the interactions that would supply the evidence never happen. The instruction is therefore methodological rather than administrative: it protects the data.
It also mirrors a general feature of relational goods, since you cannot establish whether a friendship involves trust by interrogating a friend about trust, and the same problem would undermine a journal built out of tests.
What separates a prediction that can be marked from one that cannot?
Three things: a mechanism explaining why the value should arise, an observable that an interaction could actually display, and a threshold saying how often it would need to appear. Without them a claim absorbs any evidence and cannot fail, which is the flaw markers see most often.
The additional constraint is that the value has to be one members will personally experience, rather than the most significant value for some hypothetical user, because only the first can be checked against your own transcript.
How should the mini-essay handle a prediction that was right by accident?
By naming the premise that failed rather than reporting the surprise. The requirement is to identify the problem with the underlying argument, for instance by stating which explicit or implicit premise was false, so the answer has three parts: the prediction held, the reason offered for it did not, and here is what actually produced the outcome.
Omitting the third part is the usual weakness, because without it the essay records an error instead of diagnosing one.
Exam move
Write each journal entry the same evening as the interaction, with the screenshots attached before you write the explanation rather than after. Then, once a fortnight, reread your earlier entries looking only for trends, because the later entries are expected to notice them and trends are invisible when you read one entry at a time.
Keep a separate line for any premise of your group's argument that the week's evidence has weakened.
Working through Social AI and Human Machine Relationships in PHIL7002? Sia is AskSia’s AI Arts and Humanities tutor — ask any PHIL7002 Social AI and Human Machine Relationships question and get a clear, step-by-step explanation grounded in how PHIL7002 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
