FURI | Spring 2025

Evaluating Creativity and Novelty in LLM-generated Academic Assignment Creation for Digital Design Projects

The research evaluates the creativity and novelty of large language models (LLMs) like GPT-4o, Gemini, LLaMA 3.2, and DeepSeek in generating Capstone project prompts for EEE120 within synchronous finite state machine (FSM) constraints, such as Moore models with at least five states. Using previous semester projects as a baseline, the analysis applies cosine similarity to quantify uniqueness and qualitative metrics to judge creativity and logical consistency. Outputs from agentic retrieval-augmented generation (RAG) and non-RAG setups are compared to faculty designs. The effort seeks to advance engineering education by pinpointing LLMs that deliver diverse, innovative assignments upholding academic standards.