LEE Akinobu

写真a

Affiliation Department

情報工学科 メディア情報分野
工学専攻 メディア情報プログラム

Title

Professor

Contact information

Contact information

Homepage

https://www.slp.nitech.ac.jp/

External Link

Degree

  • Ph.D. on Informatics ( Kyoto University )

Research Interests

  • Speech Recognition

  • Information Processing on Avatar Communication

  • Spoken Dialog System

  • Spoken Language Processing

  • Humanoid Agent Interaction

display all >>

Research Areas

  • Informatics / Perceptual information processing

  • Informatics / Intelligent informatics

  • Informatics / Human interface and interaction

  • Informatics / Database

From School

  • Kyoto University   Graduate School, Division of Information and Communication   Department Intelligence Science and Technology   Graduated

    - 2000.09

      More details

    Country:Japan

    researchmap

  • Kyoto University   Faculty of Engineering   Department of Information Science   Graduated

    - 1996.03

      More details

    Country:Japan

    researchmap

  • Kyoto University   Graduate School, Division of Engineering   Department of Information Science   Graduated

    - 1998.03

      More details

    Country:Japan

    researchmap

From Graduate School

  • Kyoto University   Graduate School, Division of Information and Communication   Department Intelligence Science and Technology   Doctor's Course   Completed

    - 2000.09

      More details

    Country:Japan

  • Kyoto University   Graduate School, Division of Engineering   Department of Information Science   Master's Course   Completed

    - 1998.03

      More details

    Country:Japan

External Career

  • Nara Institute of Science and Technology   Research Assistant

    2000.10 - 2005.03

      More details

    Country:Japan

  • Nagoya Institute of Technology   Associate Professor

    2005.04 - 2016.03

      More details

    Country:Japan

  • Nara Institute of Science and Technology   Research Assistant

    2000.10 - 2005.03

      More details

    Country:Japan

  • Nagoya Institute of Technology   Associate Professor

    2005.04 - 2016.03

  • Nagoya Institute of Technology   Associate Professor

    2005.04 - 2016.03

display all >>

 

Research Career

  • Information Processing on Avatar Communication / R&D on CG-specific avatar commuinication

    The Other Research Programs  

    Project Year: 2020.12 - 2025.12

     More details

    JST Moonshot R&D Goal 1 Avatar Symbiotic Society Project

  • Speech Recognition, Spoken Language Processing and Understanding, Spoken Dialog System, Voice interaction

    (not selected)  

    Project Year: 2000.10

     More details

    General topics of speech recognition, spoken language understanding, dialog systems and interactions, incorporating signal, language and perceptions.

Papers

display all >>

Books and Other Publications

Misc

  • 汎用大語彙音声認識ソフトウェア入門 Invited Reviewed

    李 晃伸

    62 ( 2 )   50 - 56   2018.02

     More details

    Authorship:Lead author   Language:Japanese   Publishing type:Article, review, commentary, editorial, etc. (scientific journal)  

    DOI: https://doi.org/10.11509/isciesci.62.2_50

    CiNii Articles

    researchmap

  • On-Campus, User-Participatable, and Voice-Interactive Digital Signage

    Keiichiro Oura, Daisuke Yamamoto, Ichi Takumi, Akinobu Lee, Keiichi Tokuda

    Academic Journal of The Japanese Society of Artifical Intelligence   28 ( 1 )   60 - 67   2013.01

     More details

    Language:Japanese   Publishing type:Article, review, commentary, editorial, etc. (international conference proceedings)   Publisher:The Japanese Society of Artifical Intelligence  

    CiNii Articles

    CiNii Books

    researchmap

    Other Link: http://id.nii.ac.jp/1004/00008160/

  • Technical Advances of Speech-Oriented Guidance System "Takemaru-kun" by 10 Years of Long-Term Operation

    Ryuichi Nisimura, Nao Hara, Hiromichi Kawanami, Akinobu Lee, Kiyohiro Shikano

    Academic Journal of the Japanese Society for Artificial Intelligence   28 ( 1 )   52 - 59   2013.01

     More details

    Language:Japanese   Publishing type:Article, review, commentary, editorial, etc. (international conference proceedings)   Publisher:The Japanese Society for Artificial Intelligence  

    DOI: 10.11517/jjsai.28.1_52

    CiNii Articles

    CiNii Books

    researchmap

    Other Link: http://id.nii.ac.jp/1004/00008159/

  • An Open-Source Toolkit Realizing Attractive Voice Interaction Systems : MMDAgent

    LEE Akinobu, OURA Keiichiro, TOKUDA Keiichi

    IEICE technical report. Natural language understanding and models of communication   111 ( 364 )   159 - 164   2011.12

     More details

    Language:Japanese   Publisher:The Institute of Electronics, Information and Communication Engineers  

    The main and unique property of a spoken language interface that attracts people is the mutual, intuitive and lively interaction via speech. In order to reveal the actual effectiveness of speech interface, the user-oriented analysis of attractiveness in spoken dialog system should be investigated in various ways and explore the technical factors that may contributes to the appearance of attractiveness through practical examinations at various systems. This paper describes development of an open-source toolkit "MMDAgent," which makes it possible to build a variety of spoken dialog systems and speech interfaces freely. The toolkit tightly incorporates the speech recognition engine "Julius" and speech synthesis tool "Open JTalk" with a 3-D CC rendering module that can manipulates modern embodied agent characters. Techniques such as on-line motion composition and HMM-based speech synthesis with speaking style-adaptive training are implemented to provide high ability to express various aspect through interaction. The interfaces and the license are designed to make the toolkit simple, flexible, and open.

    CiNii Articles

    CiNii Books

    researchmap

  • Evaluation of spotting algorithm constrained by keyword co-occurrence for dialogue systems

    Technical Report of IEICE   2010 ( 5 )   1 - 6   2011.02

     More details

    Language:Japanese  

    CiNii Articles

    researchmap

  • 音声認識ソフトウェアJulius

    河原 達也, 李 晃伸

    25   1 - 9   2011

     More details

    Language:Japanese   Publisher:人工知能学会  

    CiNii Articles

    CiNii Books

    researchmap

  • Evaluation of spotting algorithm constrained by keyword co-occurrence for dialogue systems

    KATO Aki, NANKAKU Yoshihiko, LEE Akinobu, TOKUDA Keiichi

    IEICE technical report   110 ( 356 )   25 - 30   2010.12

     More details

    Language:Japanese   Publisher:The Institute of Electronics, Information and Communication Engineers  

    Question-answering dialogue system often choose a response sentence based on recognized keywords in a user's utterance. In such case, a robust utterance understanding can be achieved by recognizing only the keywords, skipping irrelevant part of speech, rather than decoding the whole input speech. Also, since the intention of an utterance can be expressed as combination of keywords (keyword set) rather than a set of single keyword, it is desirable to extract the keywords as keyword sets. In this study, we propose an algorithm which directly applies the keyword set constraints by consulting their co-occurrences during search using a large vocabulary garbage model. By applying the constraints dynamically while search, it suppresses unnecessary hypotheses and thus expected to perform more efficiently and result in more robust intention detection. The proposed keyword-set spotting algorithm is implemented on large vocabulary continuous speech recognition decoder "Julius" fully on both passes. We evaluated the performance of the proposed method. It was confirmed that the recognition rate of keywords by spotting was superior to the dictation-based method. In addition, keyword spotting constrained by co-occurrences improved the keyword extraction rate by about 12.5 % relatively at the maximum. In this paper, we report the evaluation results in a small task of 150 keywords and the Takemaru task.

    CiNii Articles

    CiNii Books

    researchmap

  • Evaluation of Successive Rapid Hypothesis Determination Algorithm for Continuous Word Recognition

    OHNO Hiroyuki, KOJIMA Hiroshi, NANKAKU Yoshihiko, LEE Akinobu, TOKUDA Keiichi

    IEICE technical report   110 ( 356 )   77 - 82   2010.12

     More details

    Language:Japanese   Publisher:The Institute of Electronics, Information and Communication Engineers  

    Minimizing response delay of speech recognition system and giving rapid feed backs are important properties for an intuitive, easy-to-use speech interfaces. Many studies has been conducted to improve the response delay, such as making progressive outputs while recognition process "after" the words are half-determined in the context. In order to achieve higher speed input responses, we have proposed an algorithm to determine the most likely hypothesis "before" the utterance ends. The method has been examined for isolated word recognition, and this paper extends it for continuous word recognition. Experimental evaluations were performed for tasks of various vocabulary size. The result at a small vocabulary task with 14 words has shown that our proposed algorithm can determine each word for about 0.053 second prior to the actual end of speech on average, without any degradation of recognition accuracy. Another result on a station names recognition task with vocabulary size of 8738 has shown that our proposed algorithm can determine each word for about 0.48 second on average after the actual end of speech. The comparison results on various acoustic models are also reported.

    CiNii Articles

    CiNii Books

    researchmap

  • A speech-oriented information kiosk based on user-generated dialog contents

    FUKUTA Toshinori, YOSHIMI Yoshitaka, NANKAKU Yoshihiko, LEE Akinobu, TOKUDA Keiichi

    IEICE technical report   109 ( 356 )   207 - 212   2009.12

     More details

    Language:Japanese   Publisher:The Institute of Electronics, Information and Communication Engineers  

    On the development of a spoken dialog system, the system developer has to build and customize the contents for the target task. On the other hand, a Web-based "user generated" contents such as Wikipedia has been recently arisen as a new contents providing paradigm. This paper describes our trial to build a question-answering dialog system with user generated dialog contents. A user can add a response sentence freely to the system with the corresponding keywords via the Web using cellular phone or PC. The system will be updated as soon as the sentence has been added. The user can give a feedback to the system through touch panel interface, and the feedback will be reflected to the scoring of the answer sentence so that unwanted output will be suppressed. Field test for a month on public space showed the possibility of user generated dialog contents.

    CiNii Articles

    CiNii Books

    researchmap

  • Speaker Adaptation Using Nonlinear Spectral Transformation For Speech Recognition

    HAYASHI Toyohiro, NANKAKU Yoshihiko, LEE Akinobu, TOKUDA Keiichi

    IEICE technical report   109 ( 356 )   1 - 6   2009.12

     More details

    Language:Japanese   Publisher:The Institute of Electronics, Information and Communication Engineers  

    This paper proposes a speaker adaptation technique using nonlinear spectral transform based on GMMs. One of the most popular forms of speaker adaptation is based on linear transforms, such as maximum likelihood linear regression (MLLR). In MLLR, model parameters of HMMs are linearly transformed based on the maximum likelihood (ML) fashion by using a small amount of adaptation data. Although multiple transform matrices are used according to the regression class information, only a single linear transform is applied to each state within a regression class. In the proposed technique, we define a new likelihood function combining HMMs for recognition with GMMs for spectral transform and speaker adaptation based on nonlinear transform is performed in the ML fashion. In phoneme recognition experiments, the proposed technique shows better performance than the conventional MLLR approaches.

    CiNii Articles

    CiNii Books

    researchmap

display all >>

Presentations

display all >>

Industrial Property Rights

  • 音声対話システム用画像 ニルヴァ デバイスモード

    李晃伸

     More details

    Applicant:名古屋工業大学

    Application no:2022-025587  Date applied:2022.11

    Patent/Registration no:1749626  Date registered:2023.07 

    Rights holder:名古屋工業大学

    researchmap

  • 音声対話システム用画像 ニルヴァ ソーシャルモード

    李晃伸

     More details

    Applicant:名古屋工業大学

    Application no:2022-025588  Date applied:2022.11

    Patent/Registration no:1749627  Date registered:2023.07 

    Rights holder:名古屋工業大学

    researchmap

  • 音声対話システム用画像 ジェネ

    李晃伸, 石黒浩

     More details

    Applicant:名古屋工業大学

    Application no:2022-12730  Date applied:2022.06

    Rights holder:名古屋工業大学

    researchmap

  • 音声対話システム用画像 Rubica

    李晃伸, 石黒浩

     More details

    Applicant:名古屋工業大学

    Application no:2022-12729  Date applied:2022.06

    Rights holder:名古屋工業大学

    researchmap

Works

  • Avatar control software for MMDAgent-EX: Valles

    Akinobu Lee

    2024.09

     More details

    Work type:Software   Location:https://github.com/avatar-ss-cgca/valles  

    researchmap

    Other Link: https://github.com/avatar-ss-cgca/valles

  • Remdis

    東中 竜一郎, 光田 航, 千葉 祐弥, 李 晃伸

    2024.06

     More details

    Work type:Software   Location:https://github.com/remdis/remdis  

    researchmap

  • MMDAgent-EX public edition

    Akinobu Lee

    2023.12

     More details

    Work type:Software   Location:https://mmdagent-ex.dev/  

    DOI: 10.5281/zenodo.10427369

    researchmap

    Other Link: https://github.com/mmdagent-ex/MMDAgent-EX

  • CG-CA "Uka"

    Akinobu Lee

    2023.12

     More details

  • CG-CA "Gene"

    Akinobu Lee

    2023.12

     More details

  • 音声認識エンジン Julius-4.6

    李 晃伸

    2020.09

     More details

    Work type:Software   Location:http://julius.osdn.jp/  

    Julius のバージョン 4.6 を公開しました。4.6 ではDNN-HMM 計算部の GPU 対応 (CUDA) を行い、 デコーディングが3倍ほど速くなりました。そのほか、1パス文法認識への対応やバグ修正、アップデートが含まれています。 主な変更点は以下のとおりです。

    ・DNN-HMM 計算での CUDA サポート (Linux + CUDA-8,9,10 でのみ動作確認)
    ・1パス文法認識の実装
    ・Visual Studio 2017 でのビルド全面対応 (msvc/Julius.sln)
    ・修正BSDライセンスへ移行
    ・不具合の修正

    researchmap

  • MMDAgent-EX ベータ版

    2019.06 - 2023.12

     More details

    Work type:Software   Location:第33回人工知能学会全国大会およびWeb / https://mmdagent.lee-lab.org/  

    MMDAgent-EX は音声インタラクション構築ツールキット [MMDAgent](http://mmdagent.jp/) をスマートフォンに向けて拡張したアプリケーションです。キャラクターエージェントとのお喋りややりとりの内容を定義したスクリプトファイル、3-Dモデル、動作ファイルを自在に組み合わせて、エージェントと音声で会話するシステムを、誰でも構築しスマートフォンへ配信することができます。iOS、Android 用アプリのほか、各種デスクトップOS (Win/Mac/Linux) でも動作するマルチプラットフォームアプリケーションです。

    researchmap

  • 音声対話インタラクション基盤アプリ MMDAgent-EX の公開

    2019.06

     More details

    MMDAgent-EX は音声インタラクション構築ツールキット [MMDAgent](http://mmdagent.jp/) をスマートフォンに向けて拡張したアプリケーションです。キャラクターエージェントとのお喋りややりとりの内容を定義したスクリプトファイル、3-Dモデル、動作ファイルを自在に組み合わせて、エージェントと音声で会話するシステムを、誰でも構築しスマートフォンへ配信することができます。iOS、Android 用アプリのほか、各種デスクトップOS (Win/Mac/Linux) でも動作するマルチプラットフォームアプリケーションです。

    researchmap

  • Pocket MMDAgent ベータ版

    2018.09 - 2023.12

     More details

    Work type:Software   Location:日本音響学会2018年秋季全国大会 / https://mmdagent.lee-lab.org/  

    Pocket MMDAgent は MMDAgent のスマートフォン向け拡張版アプリです。Webで公開されている音声対話システムのダウンロード、サーバ側からのコンテンツ自動更新、メニュー・ダイアログ・ボタンなどのUIのサポート、汎的なログ収集・フィードバック機能を有しています。

    Pocket MMDAgentは音声対話コンテンツ再生・配信のマルチプラットフォームアプリケーションであり、無償で利用可能です。iOS 用アプリと Android 用アプリがそれぞれベータ版公開されているほか、デスクトップOS版 (Win/Mac/Linux) もあります。

    researchmap

  • 音声対話コンテンツ配信プラットフォーム Pocket MMDAgent の公開

    2018.09

     More details

    Pocket MMDAgent は MMDAgent をスマートフォンに向けて拡張した音声対話コンテンツ配信プラットフォームである。Web上で公開されている音声対話コンテンツの直接ダウンロードとサーバ側からのプッシュ更新機能、コンテンツ配信者へのログ収集・フィードバック機能を備えたクラウド音声対話システムのアプリケーションである。

    researchmap

display all >>

Other research activities

  • 音声対話インタラクション基盤アプリ MMDAgent-EX の公開

    2019.06

     More details

    MMDAgent-EX は音声インタラクション構築ツールキット [MMDAgent](http://mmdagent.jp/) をスマートフォンに向けて拡張したアプリケーションです。キャラクターエージェントとのお喋りややりとりの内容を定義したスクリプトファイル、3-Dモデル、動作ファイルを自在に組み合わせて、エージェントと音声で会話するシステムを、誰でも構築しスマートフォンへ配信することができます。iOS、Android 用アプリのほか、各種デスクトップOS (Win/Mac/Linux) でも動作するマルチプラットフォームアプリケーションです。

  • 音声対話コンテンツ配信プラットフォーム Pocket MMDAgent の公開

    2018.09

     More details

    Pocket MMDAgent は MMDAgent をスマートフォンに向けて拡張した音声対話コンテンツ配信プラットフォームである。Web上で公開されている音声対話コンテンツの直接ダウンロードとサーバ側からのプッシュ更新機能、コンテンツ配信者へのログ収集・フィードバック機能を備えたクラウド音声対話システムのアプリケーションである。

  • オープンソース音声インタラクション構築ツールキットMMDAgentの開発と公開

    2011.12

  • オープンソース音声認識エンジンJuliusの開発および公開

    2005.04

Awards

  • 委員特別賞

    2024.03   言語処理学会   大規模言語モデルを用いたEmotional Support Conversation システムの構築とその評価

    藤田敦也, 上乃聖, 李晃伸

     More details

    Award type:Award from Japanese society, conference, symposium, etc.  Country:Japan

    researchmap

  • IPSJ Yamashita SIG Research Award

    2007.04  

     More details

    Country:Japan

    researchmap

  • 電気通信普及財団 第24回テレコムシステム技術賞

    2006.05   電気通信普及財団  

    H.Saruwatari,T.Kawamura,T.Nshikawa,A.Lee,K.Shikano

     More details

    Award type:International academic award (Japan or overseas)  Country:Japan

    researchmap

  • ASJ Kiyoshi Awaya Award

    2002.04  

     More details

    Country:Japan

    researchmap

Scientific Research Funds Acquisition Results

  • Creation of a virtual space medical interview education platform using simulated patient avatars using AI-based dialogue technology

    Grant number:24H00170  2024.04 - 2029.03

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research  Grant-in-Aid for Scientific Research (A)

      More details

    Authorship:Coinvestigator(s)  Grant type:Competitive

    Grant amount:\48360000 ( Direct Cost: \37200000 、 Indirect Cost:\11160000 )

    researchmap

  • 音声対話におけるタスク完了率の最適化

    2022.04 - 2025.03

    株式会社 AI Shift 

    李晃伸, 上乃聖

      More details

    Authorship:Coinvestigator(s)  Grant type:Collaborative (industry/university)

    researchmap

  • 「しゃべって」つくる音声インタラクションシステム

    2014 - 2016

    日本学術振興会  科学研究費補助金  挑戦的萌芽研究

    徳田 恵一

      More details

    Grant type:Competitive

    researchmap

  • 超巨大データに基づくユニバーサル音声モデル構築のための技術的・社会的基盤の確立

    2013 - 2015

    日本学術振興会  科学研究費補助金  基盤研究(B)

    徳田 恵一

      More details

    Grant type:Competitive

    researchmap

  • コンテンツ生成の循環系を軸とした次世代音声技術基盤の確立

    2011.04 - 2017.03

    科学技術振興機構  戦略的創造研究推進事業 

    徳田 恵一, 李 晃伸, 南角 吉彦, 山本 大介, 打矢隆弘

      More details

    Authorship:Collaborating Investigator(s) (not designated on Grant-in-Aid)  Grant type:Competitive

    researchmap

display all >>

Other External Funds

  • コンテンツ生成の循環系を軸とした次世代音声技術基盤の確立

    2011.04 - 2017.03

    科学技術振興機構  戦略的創造研究推進事業 

    徳田 恵一、李 晃伸, 南角 吉彦, 山本 大介, 打矢隆弘 他

      More details

    Grant type:Competitive

  • 講演音声翻訳のための多言語音声合成技術に関する研究開発

    2009 - 2011

    総務省  戦略的情報通信研究開発推進制度 

      More details

    Grant type:Competitive

  • Effective Multilingual Interaction in Mobile Environments

    2008 - 2011

    European Commission  European Commission 

      More details

    Grant type:Competitive

  • ユーザ負担のない話者・環境適応性を実現する自然な音声対話処理

    2003 - 2007

    文部科学省  e-Society 基盤ソフトウェアの総合開発 

      More details

    Grant type:Competitive

Past of Cooperative Research

  • 音声対話におけるタスク完了率の最適化

    2022.04 - 2025.03

    株式会社 AI Shift  Collaboration in Japan 

    李晃伸,上乃聖

      More details

    Authorship:Coinvestigator(s)  Grant type:Collaborative (industry/university)

 

Committee Memberships

  • 電子情報通信学会   音声研究会 副委員長  

    2018.06 - 2020.03   

      More details

    Committee type:Academic society

  •   音声研究会 副委員長  

    2018.06 - 2020.03   

      More details

  • 情報処理学会   音声言語情報処理研究会運営委員  

    2016.04   

      More details

    Committee type:Academic society

  •   音声言語情報処理研究会運営委員  

    2016.04   

      More details

  • 日本音響学会   秋季研究発表会座長  

    2015.09   

      More details

    Committee type:Academic society

  •   秋季研究発表会座長  

    2015.09   

      More details

  • 人工知能学会   論文誌論文特集「知的対話システム」編集委員  

    2015   

      More details

    Committee type:Academic society

  •   論文誌論文特集「知的対話システム」編集委員  

    2015   

      More details

  • 情報処理学会   音声言語情報処理研究会運営幹事  

    2014.04 - 2016.03   

      More details

    Committee type:Academic society

  •   音声言語情報処理研究会運営幹事  

    2014.04 - 2016.03   

      More details

display all >>

Social Activities

  • ZIP-FM サマーキャンプ @ CODE FRIENDS 開催協力

    Role(s): Appearance, Commentator, Lecturer, Advisor, Planner, Organizing member, Demonstrator

    ZIP-FM / CODE FRIENDS  ZIP-FM  2019.04 - 2019.08

     More details

    Audience: Schoolchildren, Junior students, Guardians, Company

    Type:Seminar, workshop

  • ZIP-FM サマーキャンプ @ CODE FRIENDS / 名古屋市発明少年少女 開催協力

    Role(s): Appearance, Commentator, Lecturer, Advisor, Planner, Organizing member, Demonstrator

    ZIP-FM / 中京テレビ / 名古屋市  ZIP-FM  2018.04 - 2019.03

     More details

    Audience: Schoolchildren, Junior students, Guardians, Company

    Type:Seminar, workshop

Media Coverage

  • “アバター”と共生へ 体験・実験イベント 大阪 北区 TV or radio program

    NHK  NHK関西 ニュース   TV放映 https://www3.nhk.or.jp/kansai-news/20240910/2000087526.html  2024.09

     More details

    Author:Other 

    researchmap

  • 100体以上のアバターが働く「アバターまつり」--共生社会目指し実証実験 Internet

    CNET Japan  CNET Japan ニュース  https://japan.cnet.com/article/35206361/  2023.07

     More details

    Author:Other 

    researchmap

  • ロボット遠隔操作し「アバターまつり」 大阪・南港ATCで接客などの実証実験 高齢者の社会参加にも期待 TV or radio program

    朝日放送  ABCニュース  https://www.asahi.co.jp/webnews/pages/abc_20670.html  2023.07

     More details

    Author:Other 

    researchmap

  • ムーンショット型研究開発事業「アバター共生社会」プロジェクトの オフィシャルCGアバターを開発 ―誰もが自在に活躍できる次世代アバター社会の実現を目指して―

    名古屋工業大学  プレスリリース  https://www.nitech.ac.jp/news/press/2022/9607.html  2022.06

     More details

    Author:Myself 

    researchmap