SPEIT Editorial Note
At the SJTU Paris Elite Institute of Technology (SPEIT), a group of students have made steady progress in mathematical and physical studies, then achieved remarkable breakthroughs in scientific research. For example, Huang Siyuan, a second-year master's student at SPEIT, has consecutively published four papers as the first author in top artificial intelligence conferences such as NeurIPS 2023 and NeurIPS 2024. Graduated from Shanghai High School in 2019, Huang Siyuan entered SPEIT for integrated bachelor-master studies and once pursued a double degree at École Polytechnique. Currently, he is part of the LUMIA team at Shanghai Jiao Tong University, under the supervision of Associate Professor Lin Zhouhan from the School of Artificial Intelligence, and interning at the Seed Department of ByteDance. In this interview, Huang Siyuan shares the stories behind his academic papers.
✦ Q&A Highlights
✦
1
What is your current research direction? Could you briefly introduce your four AI top-conference papers?
My current research focuses on information retrieval, specifically optimizing retrieval ranking in the RAG (Retrieval-Augmented Generation) pipeline. I previously studied graph structure learning, mainly accelerating attention calculation on graphs. I feel honored to communicate with peers through papers and hope the academic community thrives. Here are brief introductions to my four papers:
· 1st paper: ACL 2025 (CCF A-class conference)
Main Title: Gumbel Reranking: Differentiable End-to-End Reranker Optimization
This work models the reranking in RAG systems as a subset sampling problem, achieving end-to-end reranker training in RAG systems by adding a learnable soft mask mechanism to the attention mechanism. It significantly improves retrieval ranking metrics and answer generation accuracy in multi-hop reasoning tasks.
· 2nd paper: NeurIPS 2024 (CCF A-class conference, Spotlight)
Title: Cluster-wise Graph Transformer with Dual-granularity Kernelized Attention
This work proposes an attention mechanism based on multi-kernel learning, featuring linear computational complexity and the ability to capture multi-granularity information, significantly enhancing performance in graph tasks.
· 3rd paper: EMNLP 2024 (CCF B-class conference, Findings)
Title: Mirror-Consistency: Harnessing Inconsistency in Majority Voting
This work introduces a consistency-based self-reflection strategy for LLM, improving the accuracy and calibration metrics of large language models in complex reasoning problems.
· 4th paper: NeurIPS 2023 (CCF A-class conference, Poster)
Title: Tailoring Self-Attention for Graph via Rooted Subtrees
This work proposes a multi-hop graph attention mechanism based on neighborhood subtree structures, with linear complexity, achieving SOTA-level accuracy in graph node classification tasks.
2
How did you determine the topics of your papers?
On the one hand, the era of large models requires reinventing many things. For example, in the field of information retrieval, traditional search engines (like Baidu) are impacted by large-model AI search (like Doubao), which often provides more accurate and in-depth answers to user queries. Search engines present an opportunity for "reinvention," so I chose to research in the direction of large-model AI search.
On the other hand, the laboratory offers ample freedom for exploration, and supervisors do not impose strict restrictions. The general research direction is often influenced by lab mates—starting to understand a field through senior students, then digging into worthy exploration points after in-depth study.
3
What insights do you have on paper writing and submission to share with junior students?
The most important insight is not to overemphasize papers. I believe papers should be treated as a "byproduct" of research, not the goal.
What truly matters is exploring fields of personal interest, trying untried methods, conducting solid experiments, and gaining valuable, original insights—these are the purposes of research. Once new insights are obtained, writing a paper is just a "byproduct" and a channel for communication with academic peers. If you treat the final publication as the goal and focus on churning out papers, you cannot truly improve your abilities, nor benefit the academic community.
4
As the first author of four top-conference papers, how do you manage time and energy to balance research, courses, and life?
I feel that with growth, I increasingly understand the saying "choice is more important than effort." In junior high, high school, and even the early undergraduate years, excelling in studies was undoubtedly the top priority. However, as I advanced to senior years and entered the workplace, I found more diverse tasks to handle and more people to interact with. With limited energy, it's impossible to perfect every task or satisfy everyone. Thus, the most critical step is identifying what truly matters. Each student has different orientations and career plans.
For me, in the early years, I focused on building a solid mathematical and physical foundation, so I dedicated all my energy to SPEIT's courses. Currently, aiming to work in the large-model field, I invest most of my energy in internships at internet companies.
5
How has SPEIT's Sino-French integrated training model influenced your research?
The Sino-French integrated model primarily helped me build a more solid mathematical foundation than other engineering majors. A series of math courses (varying by major), such as advanced algebra, enabled me to quickly understand cutting-edge AI methods and theories, empowering me to conduct higher-dimensional and more fundamental theoretical analyses in research. Additionally, the opportunity to study in France, learn with SPEIT peers, travel, and experience life together made me feel that I chose the right path.
Studying in France through SPEIT has fostered all-around growth in language skills, mathematical foundations, oral expression, research capabilities, etc.—thanks to SPEIT's rich courses that provide various opportunities.
6
Do you prefer to continue advanced studies in the future or apply research results to real-world scenarios?
After completing my master's degree, I plan to work directly. Both enterprises and universities offer fertile ground for innovation. From my experience, corporate innovation is more pragmatic and impactful, benefiting users. Currently, top domestic enterprises are becoming increasingly "tech-driven," willing to increase R&D investment in industrial contexts. I think it's necessary to analyze a company's style and one's own development plan (focus on business or research) when choosing a career. Universities, meanwhile, are more free and open. Every place has the potential to shine.
7
Could you introduce your current internship position?
I am interning at the Seed Department of ByteDance, mainly responsible for optimizing Doubao's online search algorithm. This internship is challenging and stressful, but Seed provides many valuable business problems to solve and opportunities to reconstruct search experiences and disrupt traditional search engines, which gives purpose, motivation, and a sense of value to my work.
Through this internship, I hope to engage in truly valuable work in the future—whether developing user-benefiting products or exploring emerging technologies. In short, difficult but right things bring more work motivation.
Thank you, Siyuan, for sharing your story.
We welcome more students to share their stories about “Me and My Academic Paper”!