← Back to blogs

Review of Boltly: Agentic AI Web Experience Platform

Abstract— To be honest, it is still difficult to compile modern eye-catching landing pages in the eyes of the majority. You either have decent code skills or you have to deal with a tedious, crammed no-code solution that simply cannot provide you the speed and scalability required to deliver a fully-checked production site. With the increase in size of projects, teams are juggling between the refined design mockups and the actual code, and this alternate-between, which sounds like a sequence of loops of itself. All that swapping about of context kills the brainstorming flow, combusts prototyping, slows launches and ultimately limits the response speed of team to market changes or customer feedback. In this paper I discuss a product called Boltly which is an AI-based tool that automatically renders clean, production-ready React code in response to simple natural-language queries related to layout, style, and functionality. Boltly offers the option of bridging the gap between the creative vision and the implementation of a designer to the developer by reducing the dev time without affecting the quality, performance, and scalability. It also automatically produces reusable parts and full page templates which are compatible with new frontend frameworks. Boltly has interactive workflow. Every designer, programmer and marketer can brainstorm within the same location, expose ideas, and instantly turn them into semantic code. This makes it more than avoidable of such mess as manual coding, or the inflexibility of template based systems. Throughout the adherence to the existing development standards, Boltly has the ability to provide high-speed load times, cross-browsing, and strong SEO, which removes the main technical obstacles and shortens creative productive loops. Briefly, Boltly is a new concept in a landing page design, testing and rollout, where the groups are left to use their creativity rather than the technical slow-down.

Keywords — Agentic AI Development, Natural Language Processing in Software Engineering, Autonomous Frontend Generation, Multi-Agent Software Systems

I. INTRODUCTION

Therefore, the primary goal of a landing page today is being interactive and completely appealing to the eye. Business owners, designers, even older students of our college zoom on these pages to imply a particular brand flavor and then seduce visitors to actually do something. In essence, the shared frameworks turn out to be rather inflexible, sluggish to execute, and difficult to service in the long run. In attempting to scale them, the resultant code or the so-called builds work but do not come anywhere near being considered seriously impressive, particularly when you are putting them into production on a large scale. It is the reason why so much is placed on instant feedback in academic projects. It gets the idea and the finished product on the same level and bridges the space between arguing in lecture and actually writing the code. Our paper, and the group term project, investigates how a new smart system can accept plain-text straight prompts in us and immediately process them into clean production-ready React components which immediately appear to us, enabling us to fine-tune them immediately. The trick is not merely a copy-paste script, it is an agentic AI capable of making clever design decisions without going against the principles of responsive design, the principles of accessibility, or the rules of search engine optimization.

II. PROBLEM DEFINITION

Being a student at the university, We believe that it is absolutely necessary to design successful landing pages when you are a business and other individuals creators. However, even the construction of these pages is faced with challenges that slow us down, limit our creativity as well as make the process less friendly to use. The old fashioned coding, and the new no code tools fail miserably since they are unable to turn ideas of designs into the actual websites. These problems may be divided into several main aspects:

  1. Complex Development Environment Setup: Installing a modern web-dev setup will require a fair share of technical expertise to install dependencies, configure settings and install those frameworks such as React, Node, and build tools. New initiation and old hacks spend a significant amount of time getting the compatibility and server mox fixed as well as tuning the server before they even consider the user interface or write a line of code.

  2. Lack of AI Code Generation and Testing: Man, AI technology is extremely fast at churning out small pieces of code, but it only continues to make large errors when it goes full components. The code that it coughs out is typically a mess, difficult to fix, difficult to reuse, and never much adheres to neat patterns that we are expected to live by. And, we have no hard tools to automatically verify AI-generated front-end code, thus we have to do all the heavy lifting by ourselves, such as checking performance, making it usable, and making sure it can be used on all major browsers.

  3. Fragmented Workflow: Among the problems which I can confidently claim we have all witnessed is the fact that designers and devs are living in two completely different realities. On Figma, the madness of pixel-perfect mockups is made by designers and later smacked across the team. I am always adding or removing a static image and typing in my code, trying to guess spacing or font sizes, and hoping that no one will notice.

  4. Inconsistent Design System Implementation: In our project, we were able to note that the existing tools do not quite comprehend the necessity to maintain consistency in design systems as they automatically generated components.

  5. Lack of Context-Aware Component Generation: In essence, so we can see that the available solutions are largely running in this massive contextual vacuum. Given that they never consider the reasons of what they are building, the blocks that they spit end up being generic, one size fits all blocks.

III. LITERATURE SURVEY

Having carried out a thorough literature review, I went out to fill the identified gap. We researched the fundamentals of code generation, intelligent completion systems using the newest prompt-engineering methods, a range of reasoning strategies and large language model evaluation metrics. These studies informed the design and development of my academic project, Boltly, which is an agentic AI platform to be used to build a web automation tool. All of the papers elucidated the manner in which current AI can in fact comprehend developer intent, generate both syntactically and semantically valid code and even infer complex software-engineering issues itself.

A. Code Completion and Synthesis Research

[1] CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequences

Authors: M. Izadi, R. Gismondi, G. Gousios (2022)

Such as, is the new multitoken code completion technology even attempting to make any prediction of the structural objects we behold in syntax of the programming language, as well as to make an estimate of the nomenclature conventions which are actually practiced by the programmers? We have found out that most of the codepletion systems that are available in the market are just one-trick ponies, as they either do syntax or do identifiers but not both. That is, their guesses are somewhat flimsy and fail to get the meaning that a dev might be trying to convey. The solution we used to correct that problem was to bring in a Parallel Transformer architecture. It allows us to work with Abstract Syntax Tree (AST) representations the structure of the code and the sequence of the tokens so that we can think of both structure and words. CodeFill demonstrates that blending structural and lexical information provides completions that in fact are not only intuitive and spot-on to context, but also offload developers and makes the development process faster.

[2] Domain Adaptive Code Completion via Language Models and Decoupled Domain Databases

Authors: Z. Tang, X. Chen, J. Li, Y. Liu, H. Zhang (2023)

It was our eye opener in this paper. It describes a field-adaptive approach to code completion, and it is rather good: It can dynamically adapt to specialized code completion environments and project-specific APIs. What particularly struck us was the fact that it decouples domain specific knowledge. The framework in itself unites hybrid-architectures. The recently experimented approach to that decoupled domain knowledge would enable Boltly to remain abreast of the tumultuous web practices, and yet maintain our core generation model stable and performant.

[3] Multi-task Learning Based Pre-trained Language Model for Code Completion

Authors: F. Liu, G. Li, Y. Zhao, Z. Jin (2021)

Therefore, this paper delves into the ways in which we can apply multi-task learning to trained language models and the applications of this method in code completion. They suggest a scheme which begins with a TransformerXL backbone except that they add a pointer-mixture trick in between. It means that either the model can select a token of its vocab or copy it directly based on the nearby context. This dual mode actually increases the level of accuracy of those monotonous code patterns, references to variables and API calls, which find their way into a scope and reoccur. The training itself is also cool, but several additional auxiliary tasks have been added, such as identifying identifiers, identifying syntax mistakes, and also summarizing the code. The findings point out that the model is good in handling long-range dependencies such as the maintenance of variables throughout many functions or classes.

[4] An Empirical Study on the Usage of BERT Models for Code Completion

Authors: M. Ciniselli, P. M. Compare, A. Mocci, M. Lanza (2021)

This paper was a total lightbulb to Boltly and his architecture. It essentially validated the choice of models of encoder-decoder transformers that we made. It is only important that our system, will have to first have to understand what the user is talking in plain English, and then smoke out the code, step by step. That, this paper informs us, is the gimmick semantic correspondence (finding the intent) and syntactic flexibility (making sound code). The scholars made a systematic search of code completion models based on BERT, starting at single-word scale, all the way to block scale. The conclusion is that hybrid-techniques are preferable, i.e. a bidirectional encoder (to be able to understand the context) + a unidirectional decoder (to write the code).

[5] Enriching Source Code with Contextual Data for Code Completion Models: An Empirical Study

Authors: T. Van Dam, M. Izadi, A. Van Deursen (2023)

Well, it was a big deal in this paper and it has a direct bearing on the agent-based architecture of Boltly. It actually makes the point of why do we need to keep a complete project context as we write code. Now, consider it - how can Boltly for making multi files React app? It must compute the dependencies between its components, must compute a flow of state between components and must be aware of the shared functions of utility. This study thus fundamentally ratifies the fact that there is no any other alternative method of ensuring that all new files fit in right way and hence, retaining all that context is fundamental. They excavated to degrees of neural code completion systems do better the more the source code is augmented with rich contextual metadata.

B. Foundation Models and Few-Shot Learning

[6] Language Models are Few-Shot Learners

Authors: T. Brown, B. Mann, N. Ryder, M. Subbiah, J.D. Kaplan, P. Dhariwal, et al. (2020)

Its general concept is actually the foundation of the entire design philosophy of Boltly. The summary: the article set the whole novel paradigm of the few-shot learning. It turned out that as soon as you make a model large (probably think 175 billion parameters) and impose it on a large and heterogeneous text corpus (hundreds of billions of tokens), then it just can do new things. It was capable of arithmetic, responded to questions, text writing, writing code and even simple symbolic manipulation. Few-shot method is termed as self-explanatory. You simply show some input output examples entering them directly to the prompt. The pattern is perceived and, just, applied to new cases. This is the very same paradigm of fewshots which we use.

C. Reasoning and Prompting Techniques

[7] Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

Authors: J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q.V. Le, D. Zhou (2022)

This entire Chain-of-Thought (often abbreviated as CoT) prompting concept was game-changing in our minds at university, in general; it is essentially a trick, by which big language models can be made to think rather than spit out some reckless guess. The study was exceedingly easy: the instruction being to tell the model to think step-by-step and then present its work out, suddenly it gets much more precise. It refers to quite challenging problem-solving tasks, such as multi-step mathematics, logical deduction, and even common sense dayto-day problem-solving. The article supports that with a mass of experiments using various benchmarks and the findings were uniform: CoT produced tangible benefits, particularly when dealing with exercises which involve a large number of intermediate actions. Using CoT, the agents of Boltly literally write down all their thought process, creating intermediate plans and determining what files to generate, what dependencies to install, and how their parts should communicate with one another, even before they write a single line of code.

[8] Reflexion: Language Agents with Verbal Reinforcement Learning

Authors: N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, S. Yao (2023)

So, this Reflexion paper is fundamentally the reason why Boltly can be so strong. I mean, it provides us with the whole and entire mechanism of dealing with build failures and runtime errors in the automated generation process. Consider it: when Boltly writes code which simply will not compile... or the previewing code looks buggy... this is the component which comes in force. It gets the error, cogitates on what was likely to be its cause, and produces an amended copy. What is this framework, then? It is a new approach to reinforcement learning. The entire concept is that the agent will get better when it reflects on its failures. Rather, it merely retains verbal feedback in kind of an episodic memory concerning what has gone wrong. The entire system can be seen as a mere 3-step cycle: attempt the task, verbally contemplate the failure, attempt once again, but this time with that memory.

[9] Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models

Authors: A. Zhou, K. Yan, M. Shlapentokh-Rothman, H. Wang, Y.-X. Wang (2023)

Therefore, this is our Language Agent Tree Search, LATS, a single framework that we have been borrowing and it is very impressive. Fundamentally it combines the strategic planning, logic and in reality executing something in a language agent, by stealing tricks with Monte Carlo Tree Search (MCTS). What is cool about it is that it allows an agent to examine multiple different solution paths simultaneously. It has the ability to evaluate the promise of each one of them based on these learned value functions, and then pour all its intellectual resources into the branches which are the most promising. Empirical evidence shows it all: the framework simply works much better in more complex reasoning that requires long term planning—include math theorem proving, multi step programming problems, and even strategy games. In our architecture in Boltly, these LATS principles are the ones directing the whole system of multi-agent orchestration.

[10] ReAct: Synergizing Reasoning and Acting in Language Models

Authors: S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K.R. Narasimhan, Y. Cao (2023)

Frankly speaking, with regard to the project developed by Boltly, the ReAct framework is a game-changer. It is the only thing that keeps the entire convolution between our agent of orchestration and the real world dev scenario on track. Imagine the following: as Boltly is constructing a web app, the agent continues to change gears. It does not just contemplate what code snippets it requires, but it literally drops those snippets into the sandbox there. The agent verifies the result after every movement of the agent either, in the creation of a file or the admission of a dependency. It processes into the build logs, searches for error messages and feeds back that information into the determination of the next step to work on. ReAct is a radically cool method of language-model agents to attend to things.

[11] Automatic Multi-step Reasoning and Tool Use for Large Language Models (ART)

Authors: B. Paranjape, S. Lundberg, S. Singh, H. Hajishirzi, L. Zettlemoyer, M.T. Ribeiro (2023)

I am attempting to get my brain around Boltly the ART structure, it is essentially the go to template of connecting such a language model agent with all the dev tools we require. When the agent is developing a React application, it cannot simply plop some code and walk away. It must also issue the command to install devps, build and start the dev server. This ART framework is nothing but what? The trick is that it spot-on picks and applies external computational resources where they are needed the most in a certain issue. It is based on Chain-of-Thought, but introduces an extra layer to the model, which notices when it is stuck, automatically executes the appropriate tool, and re-feeding the tool directly into the model logic.

[12] A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT

Authors: J. White, Q. Fu, S. Hays, M. Sandborn, C. Olea, H. Gilbert, A. Elnashar, J. Spencer-Smith, D.C. Schmidt (2023)

And we found this all-inclusive treatise which, in our case, was a game-changer. It is a listed way to provide sixteen types of prompt patterns of engineering—all of which have been known to work training big language models such as ChatGPT to act. The catalog in itself is relatively organised. It groups the patterns into reasonable categories. There's the Persona pattern that is just telling the model who to be just an expert. Next there is the Template pattern to persuade the output in a certain form. We translate these to humanity also, to the Fact Check pattern, which is the modeling that we confirm on our assertions in order to speak, we translate to Refinement pattern, which is the persons pretending to be the model of an answer to be iterated on.

D. Evaluation Methodologies for Code Generation

[13] Evaluating Large Language Models Trained on Code

Authors: M. Chen, J. Tworek, H. Jun, Q. Yuan, H.P. De Oliveira Pinto, et al. (2021)

Most it is interesting to note that the paper was the one that provided us with the HumanEval benchmark and had made the pass@k measure the correctness of functions the norm. HumanEval itself consists on 164 manly target-programmed problems. Each one comes having a natural language description, a function signature and test suite which is actually comprehensive and proves rightness by multiple edge cases. It wisely accepts that there are many legitimate implementations of which may exist for any given problem. It measures the probability that there will be at least 1 correct solution that will be found at k independent samples. This provides a much stronger measure of appraisal in comparison to the single-attempt accuracy measure that it ever had. That pass@k measure, directly reports to Boltly strategy of producing numerous candidate implementations and then choosing the best one of them on the basis of heuristic scoring.

[14] CodeBLEU: A Method for Automatic Evaluation of Code Synthesis

Authors: S. Ren, D. Guo, S. Lu, L. Zhou, S. Liu, D. Tang, N. Sundaresan, M. Zhou, A. Blanco, S. Ma (2020)

CodeBLEU (an automated system of quality assess, by Boltly) is honestly the key. It gives us a moral disposition towards the process of scoring the React components Boltly generates and it suggests that we do not need to have to peruse everything by hand. When our system generates a number of candidate implementations this CodeBLEU-based scoring helps us identify the actual contender of those candidates which be in accordance with the established best practices. So, what is CodeBLEU? It basically expands the classical BLEU measures in order to more effectively represent the structural and semantic properties are required to source code. This studies identified that standard BLEU which simply measures n-gram similarity between texts does not explain in any way code specific characteristics. CodeBLEU solves this by basing on a weighted score of 4 complementary measures. Abstract syntax tree (AST) structural similarity and dataflow matching are used to measure the semantic equivalence using the comparison of the usage patterns of variables.

[15] Prompt Engineering for Large Language Models

The lessons here are coming the hard way, and obtained by experiment and the practical suggestions in this latest round-up are gold. Patel and Parmar describe what they discovered works best, they provide a comprehensive context of what the task entails (having boundaries), they use clear instructions one can never get confused, and they demonstrate relevant examples of a good outcome and repeat making small adjustments in an attempt to do it better. They also depict the common failures of such models as making things up (hallucination), inconsistency or loss of direction to instructions and provide specific solutions to each of these problems.

[16] Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

Authors: L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, D. Li, E. Xing, H. Zhang, J.E. Gonzalez, I. Stoica (2023)

This paper, the main argument being proposed, is that large language models (LLMs) may be used as judges. The crux here is the fact that these models have the ability to test the goodness of other models outputs which is much more scalable as compared to having slow, and costly human scrutinizers. Two tools have been presented by the authors. One of them is MT-Bench, a multi-turn conversation benchmark which is used to evaluate the writing, reasoning, coding, math and information-retrieval capabilities of a model. The other one is Chatbot Arena where the users are given two responses from the model and are asked to perform a blind test and decide which one out of them is better.

IV. IMPLEMENTATION DETAILS AND TECH STACK

Boltly is constructed using a modern architecture that enjoys a microservice-based architecture. The configuration provides it with good scalability and performance and high fault tolerance. They can serve particular purposes remaining modular and communicating by means of clearly defined interfaces. A multilayered tech stack determines the overall workflow, whereby each element has been carefully selected in the name of being more responsive and developing more quickly.

A. Frontend

Boltly is a single-page front-end application that is modular and is developed in ReactJS and TailwindCSS. React is used to develop a reactivity of the SPA and the component structure. It is all about a dynamic, component based, super declarative approach that is natural to render, state management, and scaling patterns that cascade between a complex state of the UI. Along with the in-built sandboxed preview runtime, it is fault-tolerant, delivers a hot-module reload, and dependency installation, thus, it resolves network hiccups and maintains the end-user and developers experience continuously.

B. Agent Orchestration

It takes the intent of the user and builds whatever was determined, and it delivers concrete results. The agent translates natural language requirements into a systematic course of actions, i.e., to create a file, to execute commands etc. Reasoning Now Iterative Code editing is carried out in steps that put into consideration environmental constraints and are checked at every stage. A policy layer checks everything, makes certain it is safe and maintains dependencies clean and follows formatting convention. The loop orchestration boosts or refines its plans three or four times with the feedback of the user and on the basis of validation within the run time.

C. Backend

Since I am being a student working on the backend, I believe it should be simply a stateless API to ensure that all things remain modular and easy to deal with. The system provides event-based concurrency with the assistance of Node.js to process tons of streams of requests. We have also implemented rate limiting, structured logging, and request shaping hence performance will not reduce when traffic peaks. Strictly typed response contracts provide forward compatibility and permit the backend services to develop and remain maintainable.

D. Data and Caching

As far as data management in Boltly is concerned, we are all about speed and ensuring things are clean. The build condition is utmost temporary, by design. We will save it on the memory or on sandboxed environments this will be completely isolated and secure. The system sends the results directly to the client without having to write them to the disk in the form of a giant artifact. We make use of a caching layer that is lightweight. We have also put trust on version-pinning and integrity verification measures that ensure deterministic consistency in each single build.

E. Security, Reliability, and Operations

Security, reliability and operations are very essential to us thus I tend to consider Boltly as a student project which survives on a holistic approach of defence in depth. In essence, each code execution has a sandboxed environment with permission-scoped environments, which serve to isolate user processes and keep the system safe. To maintain the privacy of our data and ensure that we are compliant, we have scoped credentials, encrypted channels and redaction of outputs all covered. Our CI/CD pipelines are automated to smoke checks, canary roll-outs, and regression tests, therefore any type of risk that will arise during deployment is trimmed to the bare minimum. And the architecture is designed to scale horizontally and intelligent in load balancing. This is why we have a constant P95 latency to bursty prompt work overloads and when we happen to experience a burst of build events running concurrently.

V. WORKFLOW

This is the principle that the entire system is constructed upon; you tell Boltly what you want, in plain language, and then everything is handled, the planning and the implementation, and a live, working preview is provided to you. The flow is basically the way in which it functions, the moment you type the description up to the moment you export the final project.

  1. User Intent Capture: It is not only the matter of identifying the keywords but also the attempt to capture the actual intent.

  2. Project Classification: This technology base in the system will select the appropriate tech stack.

  3. Agent Planning: It has the role of breaking down that high-level request into a determinate step-by-step build plan.

  4. Template Seeding: Once this plan is locked, we inject the appropriate framework, a built-in bundler (previously most often Vite) and our default styles (Tailwind, most likely).

  5. Step Orchestration: Normalization of the instructions to an idempotent sequence is its primary task.

  6. Sandbox Initialization: We do not run bare metal code.

  7. Build construction: Once the install is received it starts the dev server with hot reloading and accurate error overlays.

  8. Live Preview: When the dev server is hot we simply point an iframe onto it and display the running application.

  9. Iterative Refinement: The user is able (and does) provide follow-up prompts such as make the header blue, add a contact form.

  10. Validation and Guardrails: All operations such as file writes to network calls go through a policy layer.

  11. Export and Handoff: The export implementation merely zips the entire build.

  12. Observability and Quality: The entire workflow is hooked with profound observability.

VI. CONCLUSION

The disconnect was real. Incidentally, since we wanted to make something that could truly collapse the gap between the moment of thinking of an idea and seeing it in the browser, we made one. The system that we have constructed allows you to write out what you desire in plain English and it will create actual production-level React. It runs in a live preview right away. On the backend we execute a multi agent system—a small crew of AIs: a certain one serves as the architect to design the component tree, another one serves as the code generator, and a third executes everything in a secure sandbox.

REFERENCES

  • [1] M. Izadi, R. Gismondi, G. Gousios, "CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequences," 2022.
  • [2] Z. Tang, X. Chen, J. Li, Y. Liu, H. Zhang, "Domain Adaptive Code Completion via Language Models and Decoupled Domain Databases," 2023.
  • [3] F. Liu, G. Li, Y. Zhao, Z. Jin, "Multi-task Learning Based Pre-trained Language Model for Code Completion," 2021.
  • [4] M. Ciniselli, P. M. Compare, A. Mocci, M. Lanza, "An Empirical Study on the Usage of BERT Models for Code Completion," 2021.
  • [5] T. Van Dam, M. Izadi, A. Van Deursen, "Enriching Source Code with Contextual Data for Code Completion Models: An Empirical Study," 2023.
  • [6] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, et al., "Language Models are Few-Shot Learners," 2020.
  • [7] J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. Le, D. Zhou, "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models," 2022.
  • [8] N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, S. Yao, "Reflexion: Language Agents with Verbal Reinforcement Learning," 2023.
  • [9] A. Zhou, K. Yan, M. Shlapentokh-Rothman, H. Wang, Y.-X. Wang, "Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models," 2023.
  • [10] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, Y. Cao, "ReAct: Synergizing Reasoning and Acting in Language Models," 2023.
  • [11] B. Paranjape, S. Lundberg, S. Singh, H. Hajishirzi, L. Zettlemoyer, M. Ribeiro, "Automatic Multi-step Reasoning and Tool Use for Large Language Models (ART)," 2023.
  • [12] J. White, Q. Fu, S. Hays, M. Sandborn, C. Olea, H. Gilbert, A. Elnashar, J. Spencer-Smith, D.C. Schmidt, "A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT," 2023.
  • [13] M. Chen, J. Tworek, H. Jun, Q. Yuan, H.P. De Oliveira Pinto, et al., "Evaluating Large Language Models Trained on Code," 2021.
  • [14] S. Ren, D. Guo, S. Lu, L. Zhou, S. Liu, D. Tang, N. Sundaresan, M. Zhou, A. Blanco, S. Ma, "CodeBLEU: A Method for Automatic Evaluation of Code Synthesis," 2020.
  • [15] H. Patel, S. Parmar, "Prompt Engineering for Large Language Models," 2024.
  • [16] L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, D. Li, E. Xing, H. Zhang, J.E. Gonzalez, I. Stoica, "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena," 2023.