Design and Evaluation of an Agentic AI Framework for Personalized Umrah Trip Planning

Umrah is a voluntary Islamic pilgrimage to Mecca, undertaken by millions of Muslims annually. Planning an Umrah pilgrimage is a complex, multifaceted process involving the coordination of travel, accommodations, and permits, yet current solutions often lack the integration of user preferences, contextual understanding, and dynamic adaptability, leaving pilgrims to navigate fragmented services manually. This paper presents an agentic AI-based architecture designed for end-to-end personalized Umrah trip planning, leveraging advanced reasoning, structured memory, and seamless integration with external services to generate efficient, context-aware itineraries. The objective evaluation, conducted using completeness score, language quality score, and cross-lingual metrics such as lexical overlap, semantic fidelity, and consistency, revealed that English itineraries tend to be more complete, whereas Arabic itineraries demonstrate higher linguistic quality. These results highlight the need for improved bilingual alignment and cross-lingual consistency in itinerary generation. Subjective evaluations, which involved LLM-as-a-Judge assessments, Panel of LLM Evaluators (PoLL), and human evaluators, further confirmed that the generated itineraries are clear, logically organized, and contextually relevant, although additional personalization and refinement of language flow are still desirable. The study also emphasizes that system performance depends strongly on model configuration, prompting strategy, and verbosity control. Balancing latency with descriptive richness remains critical for optimizing user experience across diverse pilgrim profiles. These findings highlight the potential of agentic AI for personalized religious travel planning and call for future enhancements through domain-specific reasoning, adaptive verbosity, multilingual support, and integrated data and AI governance to ensure safe, secure, accurate, and compliant data management throughout the agent’s lifecycle.