For a while now I have been laying the foundations for UI audio for the game. This was more involved than I thought.
I wanted to add audio events for everything in the game UI. From Mouseovers to mouse clicks to window opens and closes. I was not sure if all audio events would be used in the end, but I wanted everything in the UI to be able to make a sound. I also wanted everything to make a slightly different sound, even the same elements. So moving up a list should sound different than moving down a list. I figured it would make the sounds seem more natural.
This took a lot longer to achieve than I had thought of, as I wanted to pipe all audio trough Adventure Creator sound system so it would respect user volume settings. There were some limitations that took me a while to figure out and circumvent. I will not go into too much detail here, as it is pretty boring and just took a lot of time.
In the end, I created 3 different ways to play audio for the UI in the project.
- C# script for UI buttons
- C# script for AC components like inventory items & hotspots
- Custom action-list action for playing audio from triggers and events within Adventure Creator
All of these audio events would use a single audio source.
A prefab with a Constant ID which is a concept for adventure creator to be able to refer to a certain prefab in any scene. The Sound component would make sure the audio all respects the used settings. But using this Sound component added a bit of difficulty: the prefab would have to be in the root of all the scenes.
And as I wanted all my custom stuff to be under one prefab, I ended up writing some code that instantiates this GUISounds prefab for each level on level start. I could place these by hand in every scene, but I wanted to keep everything as tidy as possible and not have to drag multiple prefabs into each scene.
Instantiate audio source
if (audioSource == null)
{
Debug.Log("Audio Source missing from level, creating audio source to level");
GameObject AudioSourceObject = Instantiate(AudioSourcePrefab);
audioSource = AudioSourceObject.GetComponent<AudioSource>();
}
else
{
Debug.Log("Audio Source initialised correctly");
}This is a very simple process, but now I have a perfectly functioning audio source in each scene for UI audio.
C# script for UI audio for buttons
ButtonAudio.cs
using UnityEngine;
using System.Collections;
using AC;
using UnityEngine.EventSystems;
using UnityEngine.EventSystems;
public class ButtonAudio : MonoBehaviour, IPointerEnterHandler
{
public AudioClip _hoverAudioClip;
public AudioSource _audioSource;
float _horizontalSpeed = 0.0f;
float _verticalSpeed = 0.0f;
public float _pitch;
Vector2 _mouseVelocity;
public float _pitchRange = 0.25f;
// Start is called before the first frame update
void Start()
{
}
// Update is called once per frame
void Update()
{
//Debug.Log("initialise button: " + this.name);
}
void getPitch()
{
_horizontalSpeed = Input.GetAxis("Mouse X");
_verticalSpeed = Input.GetAxis("Mouse Y");
_mouseVelocity = new Vector2(_horizontalSpeed, _verticalSpeed).normalized;
_pitch = ((_mouseVelocity[0] - _mouseVelocity[1]) * _pitchRange) + 1;
_audioSource.pitch = _pitch;
}
public void OnPointerEnter(PointerEventData eventData)
{
getPitch();
_audioSource.PlayOneShot(_hoverAudioClip, Options.GetSFXVolume());
Debug.Log("Playing HoverAudioClip for button " + this.name);
}
}The audio for UI button hover was done with a simple C# script. This script triggers audio from the GUISounds prefab when the mouse is hovering on a button. The audio is also affected by the direction the mouse cursor is traveling to. This trick makes the audio sound more natural as you move up / down a list of options.
C# script for UI audio for AC components like inventory items & hotspots
This one is a lot more complicated. As I needed to piggyback on a lot of Adventure Creator functionality to be able to add audio to events.
I ended up plugging the functionality directly into the Hotspot Highlighter component I created a while back to drive the hotspot description UI as many of the events would be shared.
HotspotHighlighter.cs
using UnityEngine;
using System.Collections;
using AC;
public class HotspotHighlighter : MonoBehaviour
{
private string Descriptiontext = "decription";
private string DescriptionActiveItem = "active item description";
private bool itemMode = false;
public AudioClip _InventorySelectAudio;
public AudioClip _InventoryDeselectAudio;
public AudioClip _InventoryHoverAudio;
public AudioClip _HotspotHoverAudio;
public AudioSource audioSource;
bool _playAudioHotspot = true;
bool _playAudioInventory = true;
bool _PDAOpen;
float _horizontalSpeed = 0.0f;
float _verticalSpeed = 0.0f;
float _pitch = 1.0f;
Vector2 _mouseVelocity;
public float _pitchRange = 0.25f;
public Texture empty;
bool _PDAMode;
public GameObject AudioSourcePrefab;
void OnEnable()
{
EventManager.OnInventorySelect += itemHover;
EventManager.OnInventoryDeselect += itemDrop;
EventManager.OnInventorySelect += itemSelect;
if (audioSource == null)
{
Debug.Log("Audio Source missing from level, creating audio source to level");
GameObject AudioSourceObject = Instantiate(AudioSourcePrefab);
audioSource = AudioSourceObject.GetComponent<AudioSource>();
}
else
{
Debug.Log("Audio Source initialised correctly");
}
}
void OnDisable()
{
EventManager.OnInventorySelect -= itemHover;
EventManager.OnInventoryDeselect -= itemDrop;
EventManager.OnInventorySelect -= itemSelect;
}
//Select item from the inventory
void itemHover(InvItem invItem)
{
Debug.Log("Mouse over: " + invItem);
DescriptionActiveItem = invItem.GetProperty(0).TextValue;
Descriptiontext = DescriptionActiveItem;
MenuGraphic hotspotIcon = PlayerMenus.GetElementWithName("Hotspot", "Icon") as MenuGraphic;
hotspotIcon.graphic = KickStarter.cursorManager.GetCursorIconFromID(0);
MenuLabel examineLabel = PlayerMenus.GetElementWithName("Hotspot", "ExamineLabel") as MenuLabel;
examineLabel.label = Descriptiontext;
//hide the UI icon
MenuGraphic visibleIcon = PlayerMenus.GetElementWithName("Hotspot", "Icon (INV)") as MenuGraphic;
visibleIcon.SetNormalGraphicTexture(empty);
}
//Deselect the active inventory item
void itemDrop(InvItem invItem)
{
Debug.Log(invItem + " deselected");
itemMode = false;
if (audioSource != null)
{
audioSource.pitch = 1;
audioSource.PlayOneShot(_InventoryDeselectAudio, Options.GetSFXVolume());
}
Debug.Log("Playing InventoryDeselectAudio for item drop");
}
//Activate an inventory item
void itemSelect(InvItem invItem)
{
itemMode = true;
Debug.Log(invItem + " selected");
//add play audio event here for selecting items.
_PDAMode = AC.GlobalVariables.GetVariable(3).BooleanValue;
audioSource.pitch = 1;
if (_PDAMode == false)
{
audioSource.PlayOneShot(_InventorySelectAudio, Options.GetSFXVolume());
Debug.Log("Playing InventorySelectAudio for item select");
}
_playAudioInventory = false;
}
void getPitch()
{
_horizontalSpeed = Input.GetAxis("Mouse X");
_verticalSpeed = Input.GetAxis("Mouse Y");
_mouseVelocity = new Vector2(_horizontalSpeed, _verticalSpeed).normalized;
_pitch = ((_mouseVelocity[0] - _mouseVelocity[1]) * _pitchRange) + 1;
audioSource.pitch = _pitch;
}
// Update is called once per frame
void Update()
{
//Mouseuover for inventory items
if (KickStarter.runtimeInventory.hoverItem != null)
{
//Only Once
if (_playAudioInventory == true)
{
Debug.Log("hovering on inv item: " + KickStarter.runtimeInventory.hoverItem);
getPitch();
_PDAMode = AC.GlobalVariables.GetVariable(3).BooleanValue;
if (_PDAMode == false)
{
Debug.Log("Playing InventoryHoverAudio for " + KickStarter.runtimeInventory.hoverItem);
audioSource.PlayOneShot(_InventoryHoverAudio, Options.GetSFXVolume());
}
_playAudioInventory = false;
}
//Change the hotspot icon if hovering on the menu with no item selected
if (itemMode == false)
{
MenuGraphic visibleIcon = PlayerMenus.GetElementWithName("Hotspot", "Icon (INV)") as MenuGraphic;
Texture InvItemIcon = KickStarter.runtimeInventory.hoverItem.tex;
visibleIcon.SetNormalGraphicTexture(InvItemIcon);
//visibleIcon.graphic = KickStarter.cursorManager.GetCursorIconFromID(5);
//hide the hotspoticon
MenuGraphic hotspotIcon = PlayerMenus.GetElementWithName("Hotspot", "Icon") as MenuGraphic;
hotspotIcon.graphic = KickStarter.cursorManager.GetCursorIconFromID(7);
}
MenuLabel examineLabel = PlayerMenus.GetElementWithName("Hotspot", "ExamineLabel") as MenuLabel;
Descriptiontext = KickStarter.runtimeInventory.hoverItem.GetProperty(0).TextValue;
examineLabel.label = Descriptiontext;
}
//Mouseuover for hotspots
else if (KickStarter.playerInteraction.GetActiveHotspot() != null)
{
//Only Once
if (_playAudioHotspot == true)
{
Debug.Log("Mouse is over the hotspot: " + KickStarter.playerInteraction.GetActiveHotspot().name + ", playing HotspotHoverAudio");
getPitch();
audioSource.PlayOneShot(_HotspotHoverAudio, Options.GetSFXVolume());
_playAudioHotspot = false;
}
MenuLabel examineLabel = PlayerMenus.GetElementWithName("Hotspot", "ExamineLabel") as MenuLabel;
//change the hotspot UI icon if no item is selected
if (itemMode == false)
{
Hotspot _hotspot = KickStarter.playerInteraction.GetActiveHotspot();
MenuGraphic hotspotIcon = PlayerMenus.GetElementWithName("Hotspot", "Icon") as MenuGraphic;
//Get icon for the hotspot to show in the UI
int _iconID = _hotspot.GetFirstUseIcon();
hotspotIcon.graphic = KickStarter.cursorManager.GetCursorIconFromID(_iconID);
//hide the UI icon
MenuGraphic visibleIcon = PlayerMenus.GetElementWithName("Hotspot", "Icon (INV)") as MenuGraphic;
visibleIcon.SetNormalGraphicTexture(empty);
}
//Display description text if one is given
if (KickStarter.playerInteraction.GetActiveHotspot().gameObject.TryGetComponent<ExamineText>(out var examineText))
{
Descriptiontext = examineText.Description;
examineLabel.label = Descriptiontext;
}
//If no text is given, do not display any text
else
{
examineLabel.label = "";
}
}
//If holding on an item, change the text to the item description when exiting the hotspot.
else if (itemMode == true)
{
MenuLabel examineLabel = PlayerMenus.GetElementWithName("Hotspot", "ExamineLabel") as MenuLabel;
examineLabel.label = DescriptionActiveItem;
//reset audio bools for audio to play again
_playAudioHotspot = true;
_playAudioInventory = true;
audioSource.pitch = 1;
}
else
{
//reset audio bools for audio to play again
_playAudioHotspot = true;
_playAudioInventory = true;
audioSource.pitch = 1;
}
}
}This script adds audio for these events:
- hovering on top of hotspots (with pitch shift)
- hovering on top of inventory items (with pitch shift)
- activating an inventory item
- deactivating an inventory item
I am sure to add more audio hooks to more Adventure Creator features ad the project progresses, but these are the events you encounter the most.
The game does sound absolutely horrible now that everything makes a demo-beep! I urgently need to add polished, harmonious UI sounds to the game to keep my family sane!
I ended up tweaking the look of the UI as well while I was adding the audio events, making things more streamlined. Each time I ran into an issue, I posted my problems on the Adventure Creator forum Technical Q&A section and got a reply within the day. This made working on this thing so easy!
Here is an example of one of the issues that I ran into. Chris solved my problem right away. He is a super cool developer!
Custom action-list action for playing audio
The last UI audio related thing I did was a custom action for playing audio from within an action list. This would be used to add audio to things like opening up the PDA, switching PDA tabs, opening files, electing a map destination, closing the PDA, exiting the map etc. Every little thing that has some UI aspect to it that is driven by an actionlist.
ActionSoundShotCentered.cs
/*
*
* Adventure Creator
* by Chris Burton, 2013-2023
*
* "ActionSoundShot.cs"
*
* This action plays an AudioClip without the need for a Sound object.
*
*/
using UnityEngine;
using System.Collections.Generic;
#if UNITY_EDITOR
using UnityEditor;
#endif
namespace AC
{
[System.Serializable]
public class ActionSoundShotCentered : Action
{
public int constantID = 0;
public int parameterID = -1;
public Transform origin;
protected Transform runtimeOrigin;
public AudioSource audioSource;
public int audioSourceConstantID = 0;
public int audioSourceParameterID = -1;
protected AudioSource runtimeAudioSource;
public AudioClip audioClip;
public int audioClipParameterID = -1;
public override ActionCategory Category { get { return ActionCategory.Sound; }}
public override string Title { get { return "Play one-shot Centered"; }}
public override string Description { get { return "Plays an AudioClip once without the need for a Sound object."; }}
public override void AssignValues (List<ActionParameter> parameters)
{
runtimeOrigin = AssignFile (parameters, parameterID, constantID, origin);
audioClip = (AudioClip) AssignObject <AudioClip> (parameters, audioClipParameterID, audioClip);
runtimeAudioSource = (AudioSource) AssignFile <AudioSource> (parameters, audioSourceParameterID, audioSourceConstantID, audioSource);
}
public override float Run ()
{
if (audioClip == null)
{
return 0f;
}
if (!isRunning)
{
isRunning = true;
if (runtimeAudioSource)
{
runtimeAudioSource.PlayOneShot (audioClip, Options.GetSFXVolume ());
}
else
{
Vector3 originPos = KickStarter.CameraMainTransform.position;
Debug.Log("Playing audio at position: " + originPos);
float volume = Options.GetSFXVolume ();
audioSource.pitch = 1;
AudioSource.PlayClipAtPoint (audioClip, originPos, volume);
}
if (willWait)
{
return audioClip.length;
}
}
isRunning = false;
return 0f;
}
public override void Skip ()
{
if (audioClip == null)
{
return;
}
if (runtimeAudioSource)
{
// Can't stop audio in this case
}
else
{
AudioSource[] audioSources = Object.FindObjectsOfType (typeof (AudioSource)) as AudioSource[];
foreach (AudioSource audioSource in audioSources)
{
if (audioSource.clip == audioClip && audioSource.isPlaying && audioSource.GetComponent<Sound>() == null)
{
audioSource.Stop ();
return;
}
}
}
}
#if UNITY_EDITOR
public override void ShowGUI (List<ActionParameter> parameters)
{
audioClipParameterID = ChooseParameterGUI ("Clip to play:", parameters, audioClipParameterID, ParameterType.UnityObject);
if (audioClipParameterID < 0)
{
audioClip = (AudioClip) EditorGUILayout.ObjectField ("Clip to play:", audioClip, typeof (AudioClip), false);
}
audioSourceParameterID = ChooseParameterGUI ("Audio source (optional):", parameters, audioSourceParameterID, ParameterType.GameObject);
if (audioSourceParameterID >= 0)
{
audioSourceConstantID = 0;
audioSource = null;
}
else
{
audioSource = (AudioSource) EditorGUILayout.ObjectField ("Audio source (optional):", audioSource, typeof (AudioSource), false);
audioSourceConstantID = FieldToID (audioSource, audioSourceConstantID);
audioSource = IDToField (audioSource, audioSourceConstantID, false);
}
if (audioSource == null && audioSourceParameterID < 0)
{
//parameterID = ChooseParameterGUI ("Position (optional):", parameters, parameterID, ParameterType.GameObject);
if (parameterID >= 0)
{
constantID = 0;
origin = null;
}
else
{
//origin = (Transform) EditorGUILayout.ObjectField ("Position (optional):", origin, typeof (Transform), true);
constantID = FieldToID (origin, constantID);
origin = IDToField (origin, constantID, false);
}
}
willWait = EditorGUILayout.Toggle ("Wait until finish?", willWait);
}
public override void AssignConstantIDs (bool saveScriptsToo, bool fromAssetFile)
{
AssignConstantID (origin, constantID, parameterID);
}
public override string SetLabel ()
{
if (audioClip != null)
{
return audioClip.name;
}
return string.Empty;
}
public override bool ReferencesObjectOrID (GameObject gameObject, int id)
{
if (parameterID < 0)
{
if (origin && origin.gameObject == gameObject) return true;
if (constantID == id && id != 0) return true;
}
return base.ReferencesObjectOrID (gameObject, id);
}
#endif
/**
* <summary>Creates a new instance of the 'Sound: Play one-shot' Action</summary>
* <param name = "clipToPlay">The clip to play</param>
* <param name = "origin">Where to play the clip from</param>
* <param name = "waitUntilFinish">If True, then the Action will wait until the sound has finished playing</param>
* <returns>The generated Action</returns>
*/
public static ActionSoundShotCentered CreateNew (AudioClip clipToPlay, Transform origin = null, bool waitUntilFinish = false)
{
ActionSoundShotCentered newAction = CreateNew<ActionSoundShotCentered> ();
newAction.audioClip = clipToPlay;
newAction.origin = origin;
newAction.TryAssignConstantID (newAction.origin, ref newAction.constantID);
newAction.willWait = waitUntilFinish;
return newAction;
}
}
}I used the generic Adventure Creator sound playback action as a base, but changed it to work for my use case: to play centered audio trough my own prefab audio source. I might have been able to use the built in thing for this, but I ended up modifying my own version just in case.
As said, at the moment the game UI does indeed sound absolutely horrible! A mess of demo beeps and bloops! But the foundation had been laid and tested. Making the UI sound good is the fun part!
Leave a Reply