Foundation models and large language models have demonstrated impressive outcomes across diverse applications, including Chatbot design, Natural Language Interaction, Text Summarization, and more. Among the crucial components contributing to the success of these models, the Transformer stands out as a central focus of extensive research and development. We cordially invite researchers and practitioners to submit their original work to the Workshop on Foundation Models and Large Language Models. The workshop aims to foster collaboration among experts from both academia and industry, providing a platform to delve into the most recent advancements, address challenges, and explore opportunities in the realm of foundation models and large language models."

Topics of interest for the workshop include, but are not limited to:

Model Training and Optimization:

• - Techniques to deal with Hallucinations
• - Training data for LLMs
• - Efficient and Stable training of LLMs
• - Efficient Tuning Methods
• - Scalable approaches for distributed model training
• - Middleware for scale out data preparation for LLM training
• - Workflow orchestration for end-to-end LLM life cycle
• - Resource management for compute and energy efficient model training
• - Masking techniques
• - Representation learning

Model Utilization and Integration:

• - Using LLMs effectively as tools for Reinforcement Learning
• - Enhancing LLM capabilities by using external tools (Search engines, calculators, etc)
• - Visual Prompt Tuning and in-context learning
• - Enable easy experimentation with high utilization to train foundational models in the cloud.
• - Strategies to scale resources for training/fine-tuning foundational models
• - Instruction tuning including generation of instruction tuning data.
• - Parallel training: data model tensor (attention and weights)
• - Distributed workflows for data cleansing and model usage (Langchain)
• - Next arch such as sparse transformers
• - Principled AI

Application-Specific Models:

• - Math LLM
• - MultiModal Foundation Models
• - Trustworthy Foundation Models
• - Large-scale Visual Foundation Models
• - Timeseries foundation models for forecasting, prediction and control

Knowledge Incorporation and Adaptation:

• - Approaches to deal with knowledge recency to effectively update knowledge within LLMs.
• - Incorporating domain knowledge

Evaluation and Benchmarking:

• - Additional Benchmarks to fill gap between human and automatic reference-based evaluation