Collaboration diagram for EchoTrio.Director:

Classes
class	Response

class	StatusMutex
	Helper class to act as a mutex for the status value, as it may be read by multiple threads. More...

Public Types
enum	Status { Waiting , Listening , TextInput , VoiceInput }
	Current status of the director. More...

Public Member Functions
	Director ()

void	Initialise (UnityAction< Director.Response > onDirectorResponse, CancellationToken cancellationToken)

bool	IsStatus (Status value)

async void	ListenForNextUserInput (DirectorConfig config, List< string > speakers, List< string > topics, CancellationToken cancellationToken)

bool	CancelListen ()

async Awaitable< bool >	SubmitUserTextInput (string message, CancellationToken cancellationToken)

Properties
bool	EnableDebug = false `[get, set]`

bool	IsMicMuted = false `[get, set]`

bool	IsConnected = false `[get, private set]`

Private Member Functions
OpenAI.Realtime.SessionConfiguration	GetSessionConfiguration ()

async void	RecordInputAudio (OpenAI.Realtime.RealtimeSession session, CancellationToken cancellationToken)

void	InvokeOnDirectorResponse ()

void	OnServerEvent (IServerEvent @event)

void	SetStatus (Status value)

bool	TestAndSetStatus (Status condition, Status value)

OpenAI.Function	BuildTriggerResponseTool (List< string > speakers)

OpenAI.Function	BuildTriggerDiscussionTool (List< string > topics)

List< string >	ParseSpeakerOrder (string args)

string	ParseDiscussionTopic (string args)

Private Attributes
DirectorConfig	config = null

OpenAIClient	api = null

RealtimeSession	session = null

Director.Response	response = null

UnityAction< Director.Response >	onDirectorResponse = null

List< OpenAI.Tool >	tools = new List<OpenAI.Tool>()

StatusMutex	statusMutex = new StatusMutex()

string	latestItemId = string.Empty

Detailed Description

The director is the OpenAI Realtime model whose main function is to listen to the user's speech and decides the order in which the actors reply. For example, Athena can speak first, or Poseidon can speak first, or only one of them replies. The director has a secondary function of triggering a Discussion if the user mentions certain topics.

Definition at line 22 of file Director.cs.

Member Enumeration Documentation

◆ Status

enum EchoTrio.Director.Status

Current status of the director.

Enumerator
Waiting	Idling and waiting for VoiceChat system.
Listening	Listening for user input.
TextInput	Replying to text input.
VoiceInput	Replying to voice input.

Definition at line 24 of file Director.cs.

Constructor & Destructor Documentation

◆ Director()

EchoTrio.Director.Director ( )

inline

Definition at line 66 of file Director.cs.

Member Function Documentation

◆ BuildTriggerDiscussionTool()

OpenAI.Function EchoTrio.Director.BuildTriggerDiscussionTool ( List< string > topics )

inlineprivate

Create a function following OpenAI's JSON Schema for the director to trigger a discussion based on a topic. OpenAI API on function calling: https://platform.openai.com/docs/guides/function-calling

Parameters

topics The possible topics to trigger a discussion for.

Returns: The function's JSON Object.

If there is no topic given, the AI tends to hallucinate a topic. To prevent that, add a couple of placeholder discussions with the most obscure, niche topic that the user is unlikely to ever trigger.

Definition at line 399 of file Director.cs.

◆ BuildTriggerResponseTool()

OpenAI.Function EchoTrio.Director.BuildTriggerResponseTool ( List< string > speakers )

inlineprivate

Create a function following OpenAI's JSON Schema for the director to decide upon the speaker order. OpenAI API on function calling: https://platform.openai.com/docs/guides/function-calling

Parameters

speakers The names of the speakers.

Returns: The function's JSON Object.

Definition at line 372 of file Director.cs.

◆ CancelListen()

bool EchoTrio.Director.CancelListen ( )

inline

Definition at line 132 of file Director.cs.

◆ GetSessionConfiguration()

OpenAI.Realtime.SessionConfiguration EchoTrio.Director.GetSessionConfiguration ( )

inlineprivate

Definition at line 161 of file Director.cs.

◆ Initialise()

void EchoTrio.Director.Initialise	(	UnityAction< Director::Response >	onDirectorResponse,
		CancellationToken	cancellationToken
	)

inline

Initialise the director and connect to OpenAI's server.

Parameters

onDirectorResponse	The callback to invoke when the director has a response ready.
cancellationToken	Cancellation token used to cancel any async actions when the program shuts down.

Definition at line 74 of file Director.cs.

◆ InvokeOnDirectorResponse()

void EchoTrio.Director.InvokeOnDirectorResponse ( )

inlineprivate

If the director's response is ready, invoke the response callback.

Note that user transcript can come before or after the Response event!Note that user transcript can come before or after the Response event! OpenAI API Link: https://platform.openai.com/docs/api-reference/realtime-server-events/conversation/item/input_audio_transcription?utm_source=chatgpt.com

Definition at line 246 of file Director.cs.

◆ IsStatus()

bool EchoTrio.Director.IsStatus ( Status value )

inline

Definition at line 104 of file Director.cs.

◆ ListenForNextUserInput()

async void EchoTrio.Director.ListenForNextUserInput	(	DirectorConfig	config,
		List< string >	speakers,
		List< string >	topics,
		CancellationToken	cancellationToken
	)

inline

Listens for the next user input. This needs to be invoked at the start of every round, in order to let the director prepare for user input. Before invoking, you need to check that the status is Waiting.

Parameters

config	The director configuration.
speakers	The list of actors that are possibly speaking. The speaker order will be determined by choosing actors from this list.
topics	Possible discussion topics to be triggered by the user input.
cancellationToken	Cancellation token used to cancel any async actions when the program shuts down.

Definition at line 112 of file Director.cs.

◆ OnServerEvent()

void EchoTrio.Director.OnServerEvent ( IServerEvent @ event )

inlineprivate

Callback function to receive server events from OpenAI.

Parameters

event The event received from OpenAI.

Definition at line 256 of file Director.cs.

◆ ParseDiscussionTopic()

string EchoTrio.Director.ParseDiscussionTopic ( string args )

inlineprivate

Definition at line 423 of file Director.cs.

◆ ParseSpeakerOrder()

List< string > EchoTrio.Director.ParseSpeakerOrder ( string args )

inlineprivate

Definition at line 415 of file Director.cs.

◆ RecordInputAudio()

async void EchoTrio.Director.RecordInputAudio	(	OpenAI::Realtime::RealtimeSession	session,
		CancellationToken	cancellationToken
	)

inlineprivate

Definition at line 172 of file Director.cs.

◆ SetStatus()

void EchoTrio.Director.SetStatus ( Status value )

inlineprivate

Definition at line 355 of file Director.cs.

◆ SubmitUserTextInput()

async Awaitable< bool > EchoTrio.Director.SubmitUserTextInput	(	string	message,
		CancellationToken	cancellationToken
	)

inline

Submit the user text input. Used as an alternative to speaking into the microphone, usually for development & debugging purposes.

Parameters

message	The user text input.
cancellationToken	Cancellation token used to cancel any async actions when the program shuts down.

Definition at line 145 of file Director.cs.

◆ TestAndSetStatus()

bool EchoTrio.Director.TestAndSetStatus	(	Status	condition,
		Status	value
	)

inlineprivate

Definition at line 357 of file Director.cs.

Member Data Documentation

◆ api

OpenAIClient EchoTrio.Director.api = null

private

Definition at line 55 of file Director.cs.

◆ config

DirectorConfig EchoTrio.Director.config = null

private

Definition at line 54 of file Director.cs.

◆ latestItemId

string EchoTrio.Director.latestItemId = string.Empty

private

Definition at line 63 of file Director.cs.

◆ onDirectorResponse

UnityAction<Director.Response> EchoTrio.Director.onDirectorResponse = null

private

Definition at line 58 of file Director.cs.

◆ response

Director.Response EchoTrio.Director.response = null

private

Definition at line 57 of file Director.cs.

◆ session

RealtimeSession EchoTrio.Director.session = null

private

Definition at line 56 of file Director.cs.

◆ statusMutex

StatusMutex EchoTrio.Director.statusMutex = new StatusMutex()

private

Definition at line 62 of file Director.cs.

◆ tools

List<OpenAI.Tool> EchoTrio.Director.tools = new List<OpenAI.Tool>()

private

Definition at line 59 of file Director.cs.

Property Documentation

◆ EnableDebug

bool EchoTrio.Director.EnableDebug = false

getset

Definition at line 49 of file Director.cs.

◆ IsConnected

bool EchoTrio.Director.IsConnected = false

getprivate set

Definition at line 51 of file Director.cs.

◆ IsMicMuted

bool EchoTrio.Director.IsMicMuted = false

getset

Definition at line 50 of file Director.cs.

The documentation for this class was generated from the following file:

Assets/Scripts/VoiceChat/Director.cs

Classes

Public Types

Public Member Functions

Properties

Private Member Functions

Private Attributes

Detailed Description

Member Enumeration Documentation

◆ Status

Constructor & Destructor Documentation

◆ Director()

Member Function Documentation

◆ BuildTriggerDiscussionTool()

◆ BuildTriggerResponseTool()

◆ CancelListen()

◆ GetSessionConfiguration()

◆ Initialise()

◆ InvokeOnDirectorResponse()

◆ IsStatus()

◆ ListenForNextUserInput()

◆ OnServerEvent()

◆ ParseDiscussionTopic()

◆ ParseSpeakerOrder()

◆ RecordInputAudio()

◆ SetStatus()

◆ SubmitUserTextInput()

◆ TestAndSetStatus()

Member Data Documentation

◆ api

◆ config

◆ latestItemId

◆ onDirectorResponse

◆ response

◆ session

◆ statusMutex

◆ tools

Property Documentation

◆ EnableDebug

◆ IsConnected

◆ IsMicMuted