Temporal Context in Serial Order of Short-Term Memory

—If temporal contexts contribute to serial order in short-term memory is not well understood. Few studies have shown that the temporal context is related to serial order. However, contradicting results from other studies show that there is no relationship between serial recognition and temporal recognition. There is previous research showed that there indeed existed a relationship between temporal recognition and serial recognition. The current study tries to replicate the results from this study through an alternative analysis pipeline to understand the robustness of these results. Using a drift diffusion model, we try to understand the robustness and replicability of the previous research--Short-term recognition memory for serial order and timing. In results obtained from both the studies, we see that there is no relationship between temporal and serial recognition.


I. INTRODUCTION
Short-term memory is a widely studied area of research.Short-term order memory is the ability of people to recall items and their orders.There are some models like chaining models which believe people can recall items in their order because of the relationships of items with each other.And there are other models which believe that the items are recalled by their order because of their location.This is hypothesized to be based on their temporal dimension.When items are associated with each other by their temporal contexts, they can be recalled by their previous order.The current study aims to find if temporal context directly relates to serial order and time in short-term memory.
To understand the serial order, verbal memory is also important.Verbal working memory is the cognitive system which allows people to maintain speech-related information for a short time and use this information.It used to be widely believed that working memory emerged from activated long-term memory and their information.The verbal working memory is not limited to the content of long-term memory but also can work in flexible ways.The "mental whiteboard hypothesis" is used to evaluate serial context in verbal working memory [1].Verbal working memory is a well understood mechanism for the whiteboard hypothesis.The verbal information or items are stored in a specific serial order and position markers are an important element for verbal working memory.Then during retrieval of these verbal working memory, the position markers are also useful and the position markers in mental white board hypothesis are substantiated as coordinates within an internal space.The maintenance in verbal working memory can be seen as the linking mechanisms for articulatory rehearsal and attentional refreshing.Articulatory rehearsal "refers to overt repetition of the items through the inner voice" [1].Attentional refreshing brings items into attentional focus to strength memory to this information.This results in mental whiteboard hypothesis fitting well into the attentional refreshing framework.The lining up information in an internal space allows for spatial attention to scan all these items and refresh this information, in other words to enhance the memory of these verbal working memory.The mental whiteboard hypothesis can use the internal spatial cognition to meet the challenge of encoding and maintaining serial order faced by people across the verbal items.These results break the traditional idea that the verbal working memory and the spatial working memory are two separate entities [2].Working memory and verbal memory are believed to be related to the phonological loop, and spatial working memory is believed to be related to visual working memory.However, [1] showed that verbal working memory and spatial working memory can serve together.Furthermore, [3] showed that serial order is related to language-production.While it is widely believed that long-term memory is used in everyday communication, this study proposes a different take on the matter.People are used to the language they predominantly use in their daily life, and also get used to the grammar they use.Then the grammar and the order of words people would like to use was firstly believed as the information already in people's mind and can be activated as long-term memory or even seems as automatic.However, utterance planning in speaking requires serial order and considered as the one that is also activated as working memory which is part of the phonological loop which cost attention, and language has substantial memory maintenance demands.Temporal verbal maintenance and ordering outside of language system are placed in the phonological loop which is a short-term storage in working memory.The fact is that attention is needed to overcome the tendencies to reproduce past sentences and plan the message into an appropriate order to make a new sentence.List order is also believed accomplished by utterance planning which even need more attention to overcome the long-term knowledge of word orders in sentences.The information to speak does not comes as a random list.It is a coherent message which has logic and knowledge used to assemble it and helps to hold the words in order like the knowledge-word order link.But there are some similarities which would affect the order like the positional similarity-Words in close position would like to be exchanged than the words in far distances.
Syntactic constraints are also one factor which can influence the verbal short-term memory.The main part which can be influenced by syntactic constraints is the order of words.[4] tested the influence of syntactic constraints in the Temporal Context in Serial Order of Short-Term Memory Lingfei Cao ordering of word through four experiments.Multiple mechanisms about verbal short-term memory are introduced in this article.First of all, chunking mechanism is mentioned as one factor which can influence people's verbal memory.If more items can be stored in one chunk, there are more items that can be stored in total.The items that are strongly related with each other in long-term memory cost less attention to coactivate, than those are not strongly related.Re-integration mechanism is that people can use long-term memory to reconstruct their short-term memory.When the degraded information in short-term memory is ambiguous, the sequence matches the sequential constraints shown in long-term memory can be the one most likely to recall it.The composition of the whole list also seems to affect accuracy of recall of individual items.Hulme at al., 2003 founds that the greater conformance of syntactic rules can let the participants reflect a more accurate recall sequence compared to the syntactic structure within non-sentences which can thereby improve memory accuracy.It can extend to the increasing degree of syntactic regularity that can lead to increasing of memory accuracy.But the result of this experiment was inconsistent with the finding of [5] which showed that articulatory suppression did not modulate the sentence superiority effects.The response sequence was more consistent with the statistics of natural language and more syntactic than representing a systematic bias.The semantic meaning was shown to have very little impact compared to what was previously believed.There was also an effect of semantics on biasing.Syntactic biasing was stronger in the word group compared to the non-word group when the sequences poorly follow the syntactic rules.Semantic manipulation was had a strong effect on biasing, although it did not affect accuracy.Therefore, the items which form syntactically valid chunks or have been experienced together frequently in the past can be enhanced by chunking during encoding this information.By comparison to memory trace in the past or according to syntactic rules, re-integration mechanism can reconstruct the degraded memory information.Both chunking and re-integration can lead to regularization of word sequences.It is widely believed that latencies to the first item in each syntactic chunk will be longer than latencies to later items.But the result nevertheless shows nearly no difference for these two latencies.The latency data are considered to be compatible with an interaction between chunking and grouping [5].found that the syntactic constraints can influence participants' performance in short-term verbal memory through long-term representations.Therefore, long-term memory may also be considered when talking about verbal short-term memory.
Similarly, verbal memory can also be shown to be related to serial memory [6].Related the model of serial order in short-term memory with clinical applications.Short-term memory involves processing, storage and retrieval of information and allow people to recall them with their order.Timing used by participants to respond is recorded precisely by a new mechanism.The primacy and recency effects-the information in the beginning and the last of the serial are better retrieved by participants.Then the phonological similarity effect is also shown to perform similarly in this experiment.The current study would like to study the relationship between recall timing that participants use and their subsequent accuracy patterns.The authors of [6] found that the patients with mental illness were generally able to perform at a level similar to control participants.However, they generally showed slower reaction times and lower accuracy than control participants.Therefore, the mechanisms of short-time memory are not affected by illness and medication as much as people previously thought.The elapsed time also affect participant's performance as one of factors.The participants who have longer elapsed time between recall of letter will have higher possibility to miss a letter during recall.The primacy and recency effects which are normally observed in experiments are still observed in this experiment but indicated by the reaction time of participants.Participants used less time to recall the first and last items than they use to recall items in the middle.
As discussed before, the serial order in short-term memory may also be correlated with temporal contexts which helps items to associate with each other [7].The authors of answered the question about which areas of brain would like to support active maintenance of hierarchically organized temporal information by different timescales [7].The authors of found that PRC activation increased during temporal trials relative to item trials.In this experiment, they found the involvement of the PRC in temporal working memory in humans.The result of all the experiments of temporal information showed that the posterior hippocampus, the DLPFC and PRC all contribute to the online maintenance of temporal sequences.As for the item trials relative to temporal trials, the anterior hippocampus, PHC, anterior PFC and medial PFC, and retro splenial cortex all showed increased activation.In this experiment, participants were asked to actively maintain visual information or temporal information across a delay time.Then the number of to-be -maintained items were same for every trial.There are a total of three trials: Group, Position and Item.The group and position trials are mainly about the processes of temporal working memory.The group trials require participants to maintain the hierarchically organized coarse and fine temporal information.And the position trials would like to require the fine temporal information.Then the item trials need more information about the visual details in working memory.The maintenance of temporal working memory information was mainly associated with posterior hippocampus, DLPFC, and posterior parietal cortex due to their increased activation.Then the anterior hippocampus, PHC, medial and retrosplenial and anterior PFC cortex shows more activation during encoding and working memory maintenance of visual item details.Then the posterior parietal activation which are associated with maintenance of fine temporal resolution relative to maintenance of temporal information at a coarser scale.The result of this experiment is the same as what fMRI experiments demonstrate in prior experiments which show increased posterior parietal and DLPFC activation during the maintenance of temporal order during working memory.This theory was also tested in nonhuman primates.Furthermore, to find the question about which areas of brain would like to support active maintenance of hierarchically organized temporal information by different timescales, neural activity with working memory of position trials was compared against group trials.Results showed that position trials are supported by posterior parietal cortex due to the increased activation of this area.It means that this region is involved in maintaining more detailed temporal information in working memory.Therefore, the temporal process in working memory is divided into several part by the different kind of information.Then the neural basis of these processes relative to different kind of information especially the visual information is illustrated clearly be the increased activation and the different location.
To further explore the relationship between serial order and temporal context, The authors of [8] found that temporal clustering is also prevalent in serial order memory.In this research article, authors discussed three previous studies and found that in the simple strength-based chaining model, positional and temporal clustering two patterns fit better to that model than only positional clustering pattern.Therefore, this research again supports the relationship between temporal context and serial order.
However, The authors of [9] found the temporal distinctive models did not contribute to serial order in short-term memory.The temporal distinctiveness model is an important model which used the temporal context.When the items have more distinct-longer pause in the list have more opportunities to be recalled.But the event-based theories proposed that although the rehearsal process may have extra opportunities to occur after a pause, it is not benefited from temporal isolate or time.The result found that if participants did not use the pause in temporal distinctiveness model to group the list-object they have, serial recall and probed recall cannot benefit from temporal isolation.Therefore, this experiment found that if the pause of temporal isolation cannot be used to group, temporal distinctiveness model is not contributed in serial order of short-term memory.
A similar result is also found by [10].They found there was no temporal representation shown on serial order and timing task in short-term memory.To further examine these results, this study will repeat the studies of [10] but use drift diffusion model to test if the result of [10] is replicable.Based on information above, if temporal representation related to serial order is ambiguous, this experiment would aim to find if temporal representation is related to serial order in short-term memory by a drift diffusion model.

A. Participants
This experiment consisted of a convenience sample of 30 undergraduate students at The University of Bristol.Gender is not one of the variables here and hence only age of participants were collected.Participants were aged between 20 and 24.(M=21, SD= 0.85) All participants were native English speakers which results in avoidance of confounding variables.All of them needed to complete 2 different types of tasks which is order and timing with 3 different recognition probes: same, far and close.

B. Materials and Apparatus
This experiment used digits one to nine as its items in memory list.These items were presented to participants by means of a video recording.These items were spoken by a male with a neutral English accent.The frequency of items was regulated by the software Praat to 113Hz, and the length of items were controlled to be 500msec.Each trial had five digits which were randomly assigned from digits one to nine and were not repeated.There were four level of silent pauses used in this experiment: 200msec, 500msec, 1000msec and 2200msec.The silent pauses were randomly assigned from one of these four pauses.There are two different types of lures in the serial recognition task.In the order task type, the close lure is the exchange of two close items based on the original list which is given.Them the far lure is the exchange of two far distance items based on the original list.Also, the original object was shown to let participants make decisions.And in the timing task type, participants were required to find out if the items had same timing in the probe sequence as the original one.The close lure is the exchange of two items which have the smallest pause-200msec.The far lure is the exchange of two items which have the longest pause-2200msec.Each participant would receive 40 order trials and 40 rhythm trials.In each task type, participants received 20 same probes, 10 close lure probes and 10 for lure probes.

C. Procedure
Participants were tested individually in the room provided by researchers.A fixation point which indicated the beginning of trial.was presented centrally on the screen for 1000msec.Every probe was presented to participants through headphones.The task cue consisting of either an order task or rhythm task was presented centrally on the screen for 1000 msec after the presentation of the recognition probe over headphones.The participants were then asked to respond whether the presented probe is same or different with the one they were initially presented with.Both these probes were presented centrally for 1000msec.If participants believe the two probes to be different, they were instructed to press the "z" key, and if they believed the two probes to be the same, they were instructed to press the "?" key to indicate their answers.There were self-paced breaks after every 20 trials, and participants could press any key to continue the experiment.There were also six practice trials before the experiment.The practice trials consisted of three order trials and three rhythm trials, one for each type of recognition probe.

III. MODEL Drift diffusion model: Drift Diffusion model:
The drift diffusion model is applied to two-choice tasks.The evidence is accumulated from a starting point z toward one of two boundaries which are a and 0. When the boundary is reached, it means that the participants make their response to the task probe.And in the graph of diffusion model, the reaction times are relatively easy to observe.The reaction times are the time participants takes for accumulated evidence to reach one of the two boundaries and add the non-decision time.The results are easy to observe and discern as well, indicating which boundary was hit when a specific response was given by participants.As a result, is is also straightforward to obtain the accuracy parameters using the diffusion model.The boundary which is reached by the accumulation of evidence can indicate the response, and the response can in turn indicate if participants were accurate in their response.In this experiment, participants need to answer as different or same.Their answers will be the response which will then be compared to the original serial order.Irrespective of the accuracy of participant responses, the drift diffusion model will reach one of two boundaries.The value of drift rate needs to be divided at a point that reflects indifference between the two different possible choices.At this time, drift rate is zero.The zero point of drift rate can be estimated after participants are able to accumulate evidence from the stimuli.The number above the baseline (zero) will show up as positive in the graph and can be seen as positive drift rates and the number which is below the zero point will show as negative in the graph and can be indicated as the negative drift rate.Thus, the zero point is also called the "drift criterion" which can be used to decide the positive or negative drift rate of the data.This also leads to within-trial variability in the accumulation process which can also be understood as the noise indicating participant bias.It can result in items with the same mean value of drift rate at different times but in opposing boundaries.This resuts in the production of a distribution for the reaction time and also helps model the errors that were made by participants.In most cases, the drift diffusion model can fit the data successfully with both the drift rate and boundaries constant from starting point to boundaries.But in some situations, the drift rate or boundaries need to change over the course of accumulation.Even if the drift diffusion model can fit the data well, drift rate may increase over a small range at the beginning of the accumulation process.In the experiments, the boundaries, the non-decision time and the values of drift rate will be different from trial to trial.This is the across-trial variability in the drift diffusion model.In the across-trial variability, the drift rate is assumed to be normally distributed with SD η; the starting point is assumed to be consistent distributed with range sz; the non-decision component is assumed to be consistently distributed with range st.And also, the non-decision processes can be different from one task to another although they have the same stimuli.The boundaries that become sufficient shows that the decision of participants need to be under the control of the individual participants.That means participants need to make decisions individually.The boundaries can also be different distance with starting point in different situations.When participants have higher possibility of make a correct decision, the boundary make be far apart.And they can be close when participants make decisions faster.Then the starting point of diffusion model is often estimated to be the midpoint between two boundaries, but it can be moved towards the more probable boundary when the proportion of two responses are changed.

IV. RESULT
This study aims to find if temporal context can be used in serial order in short-term memory.The experiment about serial recognition and temporal recognition were designed to test the relationship between temporal context and serial order.
The drift rate is the rate of accumulated information which changes with quality of the stimulus.The drift rate of ordinal tasks in three probes are similar to each other.The close probe is higher than others which means more information is necessary to make decisions.In temporal tasks, close probes have the highest drift rate.In Fig. 1 there is a significant difference between ordinal tasks and temporal tasks.The drift rate decreased significantly from ordinal tasks to temporal tasks.Fig. 1.Mean drift rate for ordinal and temporal tasks: The plot showing the mean drift rate of three probes in two tasks.The higher the mean drift rate, the more information that is needed to make a decision.From the line graph of the boundary separation of ordinal cues (Fig. 2), the close lures have the lowest values of the boundary separation and the far lures are in the middle, the highest level of boundary separation is "same" group.As for temporal tasks, the identical lures have the highest boundary separation and far lures have the lowest separation.The tendency of boundary separation of ordinal is significantly different from temporal tasks.
Fig. 3 indicates that the main starting point of ordinal tasks is highest in far lures and lowest in "same" lures.However, in temporal tasks, far lures have lower starting point that "same" lures.The main starting point tendency of ordinal tasks is significantly different with that of temporal tasks.
From the interquartile range seen in Fig. 4, there is little difference that can be observed in ordinal task and a larger difference in temporal task.Also, the tendency of the drift rate in ordinal task is also different from hat of temporal task.
International Journal of Social Science and Humanity, Vol. 13, No. 5, October 2023 In ordinal task, far lures have the highest median, and in temporal task, close lures are the one which have the highest median.From the median of each probe types, the highest boundary separation is found in far lures for ordinal tasks and in close lures for temporal tasks.This translates to participants needing more information to make decision in ordinal tasks as compared to temporal tasks.And in ordinal task, close lures have the lowest boundary separation median.The lowest boundary separation median in temporal task is far lures.The boundary separation median in this figure showed different measures in ordinal and temporal task.
From Fig. 6, we see that the highest starting point in ordinal task are close lures, but that of temporal task are the far lures although the identical lures in ordinal task and temporal task are similar and they both have the lowest starting point median.The difference of highest starting point median still shows that the ordinal and temporal tasks have different measures.V. DISCUSSION This experiment hypothesized that the serial order in short-term memory is not directly related with the temporal representation which helps people build a bridge to related items in serial orders.Therefore, the prediction is that the result of ordinal tasks should be different from the results of temporal tasks.The result of this study supports this hypothesis.
In serial recognition, participants can distinguish the difference between the same and close probes, but not close and far probes.Conversely, in temporal recognition, participants can distinguish the difference between close and far probes but not same and close probes.This study disagrees with the hypothesis that people have their serial recognition decisions on the basis of temporal recognition.This is in contrast to the temporal distinctiveness theories.The Temporal distinctiveness theories believe the temporal separation of two items is important when participants are encoding them.The items that have greater distances between one another may have more opportunities to be recalled.The longer distances may make this item more "forced" than others.Items with shorter distance may be easier to be confused by people.Time-based models predict that the temporal displacement of items on lures influences the probability of recognition when lures are similar to the input sequence on serial recognition.However, this study does not show the influence of the temporal displacement.From the result of this study, we can see that participants distinguish the same and close lures instead of close and far lures in serial recognition.That shows that longer distances do not carry any influence in serial order.For the models which contain both temporal and positional information, this experiment shows that participants can selectively attend to temporal or position information, and they can allocate this attention during the test.
In this study, all of the participants are undergraduate students from University of Bristol who are between 20-24 years old.There is not a big age difference between participants which means the ability of memory for all of these participants are similar.Additionally, the spoken numbers are recorded from a male speaker with a neutral English accent.And all the participants are native English speakers.These procedures reduced the chances of confounding factors in this study.Then EZ-diffusion based drift diffusion model is used to calculate the unobservable data.The EZ-diffusion model is applied to two-choice tasks which make it suitable for this experiment.Participants in this study have only two choices.Thus, the data needed by EZ-diffusion model can easily be obtained in this experiment.The drift rate in this model shows that the speed with which participants make decisions, and the boundary separation is about the level of evidence participants need to make decisions.All the data used in this model can be helpful to achieve the final interpretation.However, there are also some limitations for this model.The EZ-diffusion model is not the most precisely diffusion model.However, it takes less time to get more precise results.
To explain more about the relationship between the results of this study and the time-based model, the temporal matching model of serial and temporal recognition can be used.This model is based on Ratcliff's theory of perceptual matching and the principle found in the application of perturbation theories in memory for perceptual and ordinal information.Although retrieval and encoding may add some noise, the uncertainty introduced by the time between is assumed as the noise.And this model assumes the events in sequences are on a linear temporal scale by sequence.This framework shows that the temporal dimensions is assumed by encoding the timing of the input sequence.And the order information is the only information that remains in ordinal models.The temporal matching models suggest that people can use temporal information or ordinal information for different tasks.The results of this study show consistency with the prediction of temporal matching models which means that participants primarily attend to ordinal information in serial tests and temporal information in temporal tests.As for the pure time-based model which unsuccessfully predicted the temporal dimension underlying memory for temporal and serial recognition, the event-based models [4], can be the alternative.The event-based models proposed that the order is not represented by other representations or processes.However, the dissociation between order and timing models shows the event-based models are incomplete.Then time-driven models OSCAR models proposed that the order is associated with positional maker, which is not time -based but coded for ordinal position.In all of these models, the temporal isolation effect cannot used to predict the accuracy of recall.
The findings of this study shows that the serial recognition does not affect recognition by temporal information as the temporal isolation effect predicted.This finding illustrates the relationship between these two parts of short-term memory that may help people question some theories already proposed, and it can also help people consider future ideas about the relationship between order and timing.Although serial recognition is not affected by temporal information, it does not mean they do not have any correlation.This finding cannot explain the finding of Elvevå g et al. (2004)'s that deficits in schizophrenia patients in both probed recall and duration identification removed any demanding effects.
The limitation of this experiment is that the modality effect is ambiguous in timing memory.But in this study, auditory presentation was still used.Although partial evidence supports auditory superiority effect, the result is also possible to be influenced.To solve this problem, other kinds of presentation of stimulus may be used in the future experiment.
Also, the participants are all college students and there are only 30 students who performed this study.Therefore, the sample size of this experiment is limited.If there is a larger sample size, the result may be more accurate.Also, because all participants are college students, the generalization is also limited.To solve this problem, more groups of people from different age groups are necessary to obtain results that could be more generalized.1

CONFLICT OF INTEREST
The author declares no conflict of interest.

Fig. 2 .
Fig. 2. Mean boundary separation for ordinal and temporal tasks: The plot showing the mean boundary separation of three probes in two tasks.The boundary separation is one where a response is initiated when one of the boundaries is reached.

Fig. 3 .
Fig. 3. Mean starting point for ordinal and temporal tasks: The main starting point of ordinal tasks and temporal tasks.

Fig. 4 .
Fig. 4. Distribution of drift rate for ordinal and temporal tasks: This figure shows how the drift rate of participants were distributed in different probe types for ordinal and temporal tasks.

Fig. 5 .
Fig. 5. Distribution of boundary separation for ordinal and temporal tasks: This figure shows how the boundary separation of participants are distributed in various probe types for ordinal and temporal tasks.

Fig. 6 .
Fig. 6.Distribution of starting points for ordinal and temporal tasks: This figure shows how the starting point of participants were distributed in different probe types for ordinal and temporal tasks.