Welcome to the New Media Doctoral Seminar!

Join us!

Thursday 20.02.2020 / 16:30-19:30 / Väre / R113

Abstracting and modeling experience – The view from a neurophenomenological isomorphism

In the 2020 seminar, I will look into epistemological and methodological convergence in research on media (artefacts) that combines empiricism of natural sciences with computational approaches. The line of research linking media and neuroscience has been prone to overinterpretation and misconceptions often due to the levels of abstraction and complexity regarding computational tools in neuroimaging. I will present my research that draws on the neurophenomenological programme (methodological bridging of subjective/objective gap) with an aim to underline levels of formalism between multisource data for the meaningful interdisciplinary study paradigms. These sources in my work include 1st/2nd person data (empirical phenomenology) with neuroimaging paradigms and models to characterize different approaches to brain connectivity. I have been using this conceptual and methodological framework with the specialized micro-phenomenology method to research augmented sense-making in interactive media, frameworks for affective computing and more extensively neurocinematics studies (cognitive neuroscience using films as stimuli) in order to model experience in a rigorous manner. The recent experimental work done as an Early-Stage researcher at Enactive Virtuality Lab at Tallinn University builds on previous work of understanding how film and its formal aspects constitute ecologically valid paradigm considered to approximatereal-life dynamic situations. In these terms, the perspective of expert annotation methods of stimuli, the so-called ‘stimulus error’ between the content of stimuli, experience and their brain mapping with the narrative context are addressed.

Finally, methodological aspects of this interdisciplinary approach are discussed with Dr Janne Kauttonen through his interdisciplinary expertise and research in human brain functions during naturalistic conditions using fMRI.

Introducing the guest

In 2019 review of naturalistic neuroscience and its stimuli (Sonkusare et al.Trends in CogSci), authors point out to Kauttonen et al. (2018) study: “A recent study deserves particular mention as it took advantage of cinematic style of the nonlinear narrative structure of ‘Memento’ (C. Nolan), a film that depicts successive scenes in reverse chronological order… (…). Analysis using MVPA identified patterns of activation in higher order cortical networks (precuneus, angular gyrus, cingulate gyrus, frontal poles) during presentation of key-frame scenes, which revealed contextual information about prior events.” 

“Now I know what this goes with!” – recall and narrative reconstruction in Memento

In this study we looked into how the human brain recalls and connects relevant memories with unfolding events. We hypothesized that repeating key-frames in Memento serve as immediate recall cues and would facilitate reconstruction of the story piece-by-piece. The chronological version of Memento, shown in a separate experiment for another group of subjects, served as a control condition. Using fMRI BOLD measurements and multivariate event-related pattern analysis method and representational similarity analysis, focal fingerprint patterns of hemodynamic activity were found to emerge during presentation of key-frame scenes. Despite the highly differential sensory-feature content of the key-frames, focal fingerprint patterns of BOLD activity were shared across the second presentation of the key-frame scenes (i.e., when they served as immediate recall cues) in various higher-order cortical regions. The distributed patterns of brain activity within these regions appear to underlie ability to recall relevant memories and connect them with ongoing events to facilitate understanding of what goes with what in a complex narrated story. Interestingly, there also seems to be anticipatory component in the higher-order sensory areas with the effects preceding onset of the key-frames.

BIO

Dr Janne Kauttonen

Dr Janne Kauttonen works as a staff researcher at Haaga-Helia University of Applied Sciences. His research background is strongly inter-disciplinary, including aspects from statistical physics, computational sciences, neuroscience and behavioral sciences. His particular areas of expertise are dynamical systems, analysis of large datasets, statistics, machine-learning and mathematical modelling. After obtaining his PhD in physics from University of Jyväskylä in 2012 in the field of statistical physics, he has worked as a researcher at Aalto University, Carnegie Mellon University and Laurea University of Applied Sciences. At Aalto and Carnegie Mellon, he developed computational tools and did research in human brain functions during naturalistic conditions (fMRI data) and visual system of rodents at neuronal level (2-photon imaging data). At Laurea University of Applied Sciences he studied cognitive biases of humans and voting behavior using text analytics and statistical methods. His current research at Haaga-Helia involves studying human interaction with other humans and artificial agents using physiological and behavioral measurements.

Image. Kauttonen, J., Hlushchuk, Y., Jääskeläinen, I. P., & Tikka, P. (2018). Brain mechanisms underlying cue-based memorizing during free viewing of movie Memento. NeuroImage, 172, 313-325.

BIO

Jelena Rosic, MA

Jelena Rosic is a PhD student at Aalto Media Department and Early Stage researcher at Enactive Virtuality Lab, Tallinn University. She has a background in professional film editing (&dramatic arts studies), cognitive film studies with embodied perspective and neuroaesthetics of film. Her interdisciplinary research combines empirical phenomenology and enactive cognitive sciences to study embodiment in the spirit of neurophenomenology (Varela 1996). She has specialized in the second-person methods, in particular, the micro-phenomenological interviewing and analyses method (trained with Prof Claire Petitmengin,) to explore usually unrecognized or inaccessible dimensions of lived experiences. Her micro-phenomenological research in the context of augmented sense-making has focused on identifying the conceptual and empirical groundings of meta-perceptual modalities that are reported to emerge in lived experiences of technologically enhanced embodiment. As a member of the NeuroCine research group led by Professor Pia Tikka at Aalto University, she gained several years of experience in conducting interdisciplinary research within neurocinematics, a line of research that applies cognitive neuroimaging methods to study intersubjectively shared narrative experiences of film viewers. In the NeuroCine she participated in developing systematic annotation methods of film stimuli for optimising the linking of cinematic features to fMRI data. As a PhD candidate at the Crucible Studio, a research group at Aalto University that investigated interactive media combining theoretical and empirical methods, she applied her neurophenomenological approach to the study of enactive narratives. Jelena’s current research at Enactive Virtuality Lab applies the micro-phenomenological method to provide understanding on the lived experience of narrative sense-making in the context of movie viewing and engaging immersive environments.

Welcome all to the Journey to the Gulag / 30.01.2020!

Welcome everyone to the third New Media Doctoral Seminar of the semester! The presentations are open to everyone. Please join us next Thursday!

Thursday 21.11.2019 / 16:30-19:30 / Väre / R113

Additional to the programme, Annukka Jyrämä (Senior Advisor, Research ethics at Aalto) will begin the session with a brief presentation on research ethics and Aalto’s processes and services.

Led by Professor Lily Diaz presentations to follow:

 Reflections on making spatial design tools / Tero Heikkinen

Abstract:

The use of artistic and design processes as part of research has provided exciting realms to explore for researchers in the art and design fields, but these approaches have also brought challenges. In this presentation I take a reflective look back at one practice-led research process. How does the understanding of space become revised for a designer, through building tools and concepts, if this process is seen as a creative project explored through multiple prototypes and tasks? The prototype cases suggest further prototypes and literature, in an “iterative cyclic web” of practice-led research that here alternates between digital software and material tools. How to transform the sometimes chaotic and nebulous experiences of creative processes into a manageable research project, material for study and shareable outcomes? I discuss one way practical cases have become intertwined with each other and what choices informed them.

Biography: 

Tero Heikkinen is a postdoctoral researcher at the Center for Artistic Research at the University of the Arts Helsinki. As a researcher he is interested in approaches that combine creative productions, design processes or artistic activities. He currently researches the uses of media-rich web platforms for artistic research output and publication. He graduated as a Doctor of Arts in design from Aalto ARTS in 2013, where he previously studied Furniture and Spatial Design.

‘Place’ in Internet of Things / Neha Sayed

Abstract:   

‘Internet of things’ promises a new configuration of space augmented with data sensing and sharing technologies challenging the notion of ‘PLACE’. The advent of these surveillance technologies promises an ‘openness’ affecting the way we live our daily life. This research aims at exploring the evolution of this place in ‘Internet of things’.Building on the Posthuman Ontology of Karen Barad, I began research by trying to investigate the notion of an architectural place. I designed a methodological framework which is multidisciplinary consisting of four disciplines, architecture, urban design, urban planning and anthropology. The research is autoethnographic and the place selected for study is a market square where last five generations of mine have resided and practiced trade. During 2016-17, I conducted observations with the community on the market square. In the beginning of 2019, along with the community I designed a concept for traffic regulation using sensors for Panasonic Design Competition. The actual design is to be tested within the community. This research presents a meaning of place and attempts to define the change caused by the sensor driven technologies.

Biography:

Basically an architect from Mumbai Neha has done a combination of architectural practice and teaching for the last eighteen years. She did her Masters in Experience Design from Konstfack, Stockholm, which added another skill of being a researcher. Her work in the Masters was focused on developing user-centred design and studying research methodologies to achieve the expected outcome. Since 2009, apart from teaching and architectural practice she has been conducting research within communities and their relationships to space and in turn place. In 2012-13, she lead a team of researchers to draft street furniture manual for a heritage town of Matheran, which is the only pedestrian tourist destination in India having a very unique community dynamic. The manual is being adopted for the policies and design. The project inspired her to think of place specific design interventions with active participation of the community. Her practice as a designer has remained strong all this while, which is mostly around interior architecture, where the smart technologies are treated as a material. Her concern about the role of sensor driven environments grew and it led her to explore the changing nature of place in the case of Internet of Things.

Accelerate Finland:
Open Call for Entries

Deadline 4 November, 2019

Arts at CERN<https://arts.cern/>, in Geneva, in partnership with the Finnish organisation Capsula<http://www.capsula.fi/> (art-science-nature), is offering an individual artist or artistic collective the opportunity to apply for a one-month research residency at one of the world’s largest particle physics laboratories, CERN.

Accelerate Finland is an open call for Finnish artists interested in carrying out artistic research inspired by the world of particle physics, in dialogue with scientists, engineers and staff, during an artistic residency of one month at CERN, the European Laboratory of Particle Physics in Geneva, Switzerland.

The main goal of the Accelerate programme is to foster artistic investigation in connection with scientific research and fundamental physics. A jury of experts from culture and science will review the proposals received out of the call. One individual artist or collective will be selected from all the received proposals and will be invited to CERN in 2020 to explore possible new scenarios for enquiry in arts and science.

Saastamoinen Foundation<http://saastamoinenfoundation.fi/> and and The Committee for Public Information<https://www.tjnk.fi/> (TJNK) support Capsula for the Accelerate Finland residency award, a collaboration with Arts at CERN.

For more information, please enter here<https://arts.web.cern.ch/open-entries/accelerate-finland>.

Welcome everyone to the second doctoral seminar of the autumn semester!

On the 24.10.19, 16.30-19.30 at Väre R113, led by Professor Lily Díaz doctoral candidate Afdila Mamdooh will present her on-going research titled

 “The possible future of films as a therapeutic and diagnostic tool”.

The use of films as stimuli in experimental settings is gaining popularity in the science community due to their unique ability to transport the observers into the world depicted in the film and tap into both unconscious and conscious behavioral, psychological, physiological and neural mechanisms. However, as films were not designed to be used for experiments, they lack the flexibility needed to allow the researchers to modify them according to their needs. Emerging evidence from the computer science field show that artificial intelligence might help overcome these limitations. In this talk I will present briefly the uniqueness of film as an art form to simulate social perception and propose how films when combined with artificial intelligence (capabilities) can serve as a therapeutic and diagnostic tool. I will also discuss the ethical aspects of pushing the boundaries of movies and how that will affect it as an art form. 

Bio:

Mamdooh Afdile, is a filmmaker and researcher in media and neuroscience, using physiological measurements and brain imaging techniques to study audience perception and social bias. Currently, he is an assistant professor at the film and media department in Stockholm university of the arts.

Work image

Guest speaker

 Iiro Jääskeläinen

 “Using movies as real-life like stimuli during neuroimaging to study the neural basis of social cognition”

Movies and narratives are increasingly used as stimuli in neuroimaging studies. This in many ways helps bridge the gaps between neuroscience, psychology, and even social sciences by allowing stimulation of, and thus also measurement of neural activity underlying, phenomena that have been less amenable to study with more traditional neuroimaging stimulus-task designs. As recent examples of our work that I will go through in my talk, observation of signature patterns underlying discrete emotions across largely shared brain structures have suggested that both basic and dimensional emotion theories are partly correct. Robust differences in brain activity when viewing genetic vs. adopted sisters going through a moral dilemma in a movie clip have shown that knowledge of shared genes shapes perception of social interactions, thus demonstrating how neuroimaging can offer important measures for social sciences that complement the traditional behavioral ones. Further, more idiosyncratic brain activity has been observed in high-functioning autistic than neurotypical subjects specifically in putative social brain regions when watching a drama movie. I will also present some results on our most recent study in preparation wherein we have observed that family ethnic-cultural background shapes brain activity and associations elicited during listening to an audiobook. Development of data analysis algorithms holds keys to rapid advances in this relatively new area of research. Modeling the stimulus and recording brain activity is significantly complemented by behavioral measures on how the subjects experienced the movie stimulus.

Human Brain Mapping magazine, 38/7, 2017 Differential human brain activity recorded during watching of sections from three movies consecutively vs. in an interleaved fashion as when watching a parallel narrative

BIO

Iiro P. Jääskeläinen is a tenured Associate Professor at the Department of Neuroscience and Biomedical Engineering at Aalto University School of Science. In his research, movies and narratives are used as naturalistic stimuli during neuroimaging to study human social cognition and emotions. He received his PhD at the University of Helsinki in 1995 in psychology, subsequently worked as an Instructor in Harvard University in Boston, MA, prior to returning to Helsinki University of Technology, the predecessor of Aalto University. Jääskeläinen has more than 130 publications in international peer-reviewed journals including PNAS, TINS, Biological Psychiatry, and Nature Communications.

Welcome everyone to the first doctoral seminar of the autumn semester!

On the 26.09.19, 16.30-19.30 at Väre R113, led by Professor Lily Díaz doctoral student Jana Pejoska will present her on-going research titled

New Media Design Considerations in Technology-enhanced Learning

Enhancing Communication and Collaboration Through Embodied Experiences

Designers of technology-enhanced learning (TEL) typically investigate how to design technology as an effective vehicle for learning. One of their challenges is to approach design which draws both from knowledge about human cognition as well as from studies that inform about the role of technology in the processes of learning. TEL has been studied for facilitating digital and multimedia literacy; for assessing the learners; for accessing and studying the learning material; as media for learning through inquiry; as media for learning through construction; or as media for learning through communication and collaboration.

In this study, I approached the design investigations in TEL for enhancing communication and collaboration practices for informal learning at work. In this field, I contribute with a design proposal using post-phenomenological theory, embodied experiences and enaction as theoretical approaches to learning, in addition to context-dependent learning paradigms.

I present our two design cases for TEL in physical work contexts, made with a design thinking approach and a research-based design method. With a research-based design method I formulate a hypothesis in the form of prototype, a result of an iterative process arising from contextual inquiries and participatory design workshops. The design outcome, the prototype for an augmented reality application for smartphones, is discussed in the light of improving knowledge sharing through remote communication and collaboration.

In addition to these cases, I discuss other outcomes of these studies, which changed the course of my investigation of design of TEL, towards critical design considerations for technology. Finally, I present Culture Coding, a new design framework for unconventional thinking towards the codes of behavior that guide human-technology relations.

Bio

Jana Pejoska is a doctoral candidate at the Media Department in the School of Art, Design and Archi at Aalto University. Currently, she is in her 4^th year of study and research connected to the Learning Enviroments Research Group, where she has been designing, developing and investigating new media prototypes such as video-conferencing interfaces, wearable device based on body memory, augmented reality for enhancing remote collaboration and informal learning, as well as a virtual reality application design for personal growth. She has a master’s degree in Digital Culture from the University of Jyvaskyla in the topic of educational virtual worlds for children, and a professional background of 15 years in digital media production such as 3D animation,  and design and development of serious games.

Guest speaker 

Georgia Panagiotidou

Using Data Visualisation for Interdisciplinary Sensemaking

Interdisciplinary collaboration describes a situation in which two or more scientific disciplines working on related problems try to interoperate to come to common understandings. In light of past shortcomings to address complex societal issues, interdisciplinary collaboration is currently employed to promote innovation and to discover holistic solutions. At the same time, ten years into the fourth wave of science, computational methods and data visualisation analysis have been established in the practise of most research paradigms from natural sciences to the humanities. Still, data collaboration comes coupled with epistemological tensions related to misalignment of interests or scientific paradigms; but also very practical tensions related to infrastructure, methodologies, ownership and even representation. My work investigates how such data issues manifest within an archaeological research project and subsequently how we can design or adapt visualisation infrastructure to address them. By organising a series of 10 interventions divided between disciplinary/interdisciplinary and data/theory-driven I was able to approach the first sub question and see how data is being negotiated in practical settings. Moreover, in an attempt to externalise the collaboration to third parties, I attempted to capture the thematic variation of the interventions as evolving node-link diagrams. Overall, my findings so far, indicate that many of the divides in data-driven interdisciplinarity are parallel to metadata and contextualisation discussions taking place in digital humanities and critical data studies. In this presentation I will discuss my methodological choices in light of the findings as well as outline how I intend to continue on the second half of my research endeavour.

Bio

Georgia Panagiotidou is a Ph.D researcher in KU Leuven where she is a member of the Research[x]Design group and the Sagalassos Archaeological Research project. Her work focuses on the design and appropriation of data visualisation for cross-disciplinary collaboration and specifically in the digital humanities. She holds a BSc. in Informatics and Telecommunications from the National and Kapodistrian University of Athens (2011) and a MA in New Media from Aalto University (2015). After graduation and before starting in KU Leuven, she worked in the field of civic technology as a data specialist for organizations including Open Knowledge Finland and the Finish newspaper, Helsingin Sanomat.

Welcome to HELDIG Forum on Monday 23.9.2019 at 14:15 with a talk by
Prof. A. Aneesh from Sociology and Global Studies, University of
Wisconsin, Milwaukee (https://uwm.edu/sociology/people/aneesh-a/). The session takes place in in Metsätalo Hall 2 (spiral staircase up from the lounge, on the right). The topic of Aneesh’s presentation is:

“Big Data and Identity Construction”

Abstract:

“While social identity is an identity continually renegotiated through linguistic interactions and social performances, bureaucratic identity – glimpsed in passports, driver’s licenses, and other identity cards – is a construction of fixed personhood for the purposes of modern organizational needs, ensuring that the member has remained essentially the same despite changes in personality, body, and behavior. With the spread of digital technologies, however, there has emerged a new variation of identity—system identity, which represents persons as dynamically forming clouds of data. While system identities can serve the bureaucratic need for identifying members, their role far surpasses the organizational necessities of inclusion and exclusion. The notion of system identities posits a space of identity construction that does not have a substrate or foundation on which identity is based. It describes an operation in which various systems – financial, legal, medical, governmental or any number of others – incorporate their own foundations by adapting themselves to their own results. This presentation highlights the importance of this differentiation and charts its latest development.”

HELDIG Forum sessions are open and free for everybody to join. These meetings are educational and dissemination activities targeted especially for researchers and teachers willing employ digital methods more in their work. Welcome!

Coffee is served at 14:00-15:00 at Metsätalo, Unioninkatu 40. See the programme details and the registration form below – please register so that we can order the right amount of coffee:

http://heldig.fi/forum