The Effect of Laser Cutting Heuristic Presentation Modality on Design Learning

With the advent of campus makerspaces and their recent, widespread integration into engineering curricula across the United States, the skills of design and prototyping have become vital to success in the academic and professional spheres. Laser cutting, previously restricted mainly to professional job shops, is one fabrication method that has unique design considerations. Some of these include compensating for the kerf, or cutting width of the laser; the fact that laser cutting is a 2D cutting process; and ensuring that the material being cut does not catch fire due to excessive heat buildup. In the academic space, it can be challenging for novice designers or students to understand these considerations. This is because traditional fabrication methods, such as manual machining, drilling, and using basic hand tools, have been taught for a comparatively much longer time than laser cutting, so while there is some research into laser-cutting processes [1–5], the same has not been extended to design guidelines to use in the academic space. Furthermore, most knowledge about laser cutting is not in formal academic research, but rather scattered across the maker community via online forums, blog posts, and videos. Exposure to formalized knowledge about other manufacturing methods, such as additive manufacturing [6], has been shown to improve novices’ ability to create designs that are well-suited for that method. When novice designers encounter a particular problem, they can make use of heuristics, or rules of thumb, to broaden their thinking and generate more varied solutions than they could come up with alone [7]. Since formalized heuristics for laser cutting could not be found, researchers generated them by consulting design blogs and experts at academic makerspaces, and made use of prior knowledge as well.

The goal of this paper was to quantify students’ ability to understand heuristics for laser cutting when the heuristics were received in different modalities. Prior research shows that presenting information in different modalities can impact understanding and retention of the material [8]; however, this knowledge has not been applied to heuristics for laser cutting. By studying the effect of presentation modality on the understanding of laser-cutting heuristics, this paper will provide further clarity on how fabrication processes can be presented in the classroom and industry settings.

Based on the goals stated previously, this paper will address the following research question:

The question addresses a few important aspects of design cognition and education. One aspect is whether it is better to present new knowledge in a text-only format or with a visual or tactile aid. The other aspect is how this knowledge affects the design outcomes—quality and novelty—of resulting designs. It was expected that a text-only modality would be the most open to interpretation and most difficult to understand, resulting in lower quality but higher novelty designs. On the other hand, presenting with tactile aids shows an exact example of the heuristic in action, which was expected to result in lower novelty, but higher quality designs. This balancing effect between quality and novelty with respect to prior knowledge is also crucial to understand in the context of design education.

When facing any open-ended problem, designers can be restricted by the pre-conceived ideas and biases they possess to creating a solution, which means they may fail to consider other potential solutions that may be better suited to the design task at hand; this is often referred to in the literature as design fixation [9]. This problem is especially prevalent in engineering students due to a lack of instruction in strategies for ideation and concept generation [10]. Heuristics, often referred to as rules of thumb, can help broaden the scope of a designer’s thinking and combat this problem of fixation. Fu et al. [11] defined heuristics as “a context-dependent directive, based on intuition, tacit knowledge, or experiential understanding, which provides design process direction to increase the chance of reaching a satisfactory but not necessarily optimal solution”. It is important here to note the distinction between heuristics and mathematical optimization. While optimization attempts to find the best solution to a problem, heuristics attempt to find a “good enough” solution [12]. Heuristics are useful in open-ended design problems since these problems tend to involve balancing multiple engineering constraints, making optimization challenging [12].

Heuristics can be a powerful tool in the design process, especially in concept generation, where designers attempt to generate as many varied ideas as possible to fulfill a particular design problem. In this early design phase, it is important to generate a wide variety of ideas, as this will increase the likelihood of finding a particular solution that matches all the required specifications of the design problem and can be further developed toward a final product [13]. Studies have shown that only 8% of costs associated with the design of a product originate from the early concept generation phase; however, these decisions are responsible for about 70% of the associated cost over the lifetime of the product [14]. Therefore, it is important to understand how designers can successfully generate multiple concepts and avoid fixation in the concept generation stage, and there is extensive literature focusing on this topic. Yilmaz et al. [15] identified heuristics that an expert designer applied throughout a two-year project and discovered that the designer was not consciously thinking about these heuristics while generating concepts. It seems to be that the knowledge these experts have is implicit and based on their prior experience, rather than something that experts consciously think about while designing. Effective design heuristics can also be derived from award-winning product designs as demonstrated by a different study by Yilmaz et al. [14]. Another pattern observed in expert designers is opportunism, which is when a designer combines two functions of a product together [13]. From this prior research, it is clear that design experts are able to apply innate experience from previous work they have done in order to foster creative thinking when generating concepts for the new problem they are working on.

While researchers have studied how design experts successfully generate novel concepts for a design problem, as discussed previously, it is also important to look at novice designers, such as engineering students, to understand the challenges they face due to their lack of experience, and how heuristics affect their design thinking. It has been found that novice designers face problems generating multiple ideas to a design problem; they may become fixated on the first concept they think of despite its flaws, or they lack the awareness of strategies used to foster creativity that experts possess [16]. More specifically, students in the classroom are often recommended brainstorming as an ideation tool. However, students are not always given clear directions when brainstorming; rather, they are told to pursue whatever comes to mind at the moment, which means that it is common for students to think of an idea without pursuing any specific ideation strategies and become fixated on that idea from the start [17]. As opposed to brainstorming, the Design Heuristics method presents rule-based approaches to systematically enhance creativity when generating design concepts [18]. Through the aforementioned experiments of collecting heuristics from expert designers and award-winning product designs, a set of 77 flashcard-style heuristic prompts was developed and distributed to students along with an ideation problem [19]. The study found that students who successfully applied these heuristics generated more novel and creative ideas [20].

The processes of laser cutting and laser engraving are two methods to apply laser energy to the material. In laser cutting, the energy is used to repeatedly vaporize a thin line of material until it is fully cut through. In laser engraving, the beam scans back and forth over the material to remove a thin depth of material from the surface. The shape of the engraving can be controlled by the user. Although laser cutters are considered a “new” technology, the technology of gas-assisted laser cutting dates back to May 1967, when assist gas and a focused laser were combined in an experiment by Peter Houldcroft, Deputy Scientific Director at The Welding Institute [2]. Like other rapid prototyping methods such as 3D printing, laser cutting has experienced a recent surge in popularity, in both the hobbyist and academic realms. One reason for this rise in popularity is the need for rapid prototyping as part of engineering students’ senior design or capstone design courses. For example, Carnegie Mellon University [3] and KU Leuven [4] have integrated laser cutting into curricula on modern making skills to enable them to create proof-of-concept prototypes.

While there is an interest and demand for engineering students to learn rapid prototyping and making skills, teaching practices can be ineffective, and these skills tend to be underutilized in the classroom; the same holds true for laser cutting [5]. To address this gap and assist novices in learning digital fabrication practices effectively, heuristics, as discussed previously, can be used to increase the likelihood of generating successful designs. A number of studies have been performed in order to generate formal heuristics for additive manufacturing, as well as demonstrate the value of these heuristics in engineering design [6,21–23]. However, there is very little formal information in the academic literature about best practices for laser cutting. Rather, best practices around laser cutting were found in online sources, such as design blogs or forums operated by hobbyists and makers. The researchers in the present study used this knowledge to generate formalized heuristics for laser cutting, which serve as a key contribution of this paper, and are presented in Appendix  A. The following explains the derivation and development of each heuristic in detail.

The second author in this study had prior experience in laser cutting through volunteering at the Invention Studio, an on-campus makerspace at Georgia Tech. One common pitfall that users of the space experienced with the laser cutters occurred when transferring their designs from a computer-aided design (CAD) program to Inkscape, the 2D graphics tool used to prepare designs for the laser cutter. Specifically, users would sometimes encounter an error in Inkscape that would cause the design dimensions to scale up or down by a random amount so that they no longer matched the intended dimensions from the CAD software. One best practice that the volunteers in the Invention Studio came up with was to ask users to check one key dimension in their design after importing it into Inkscape. Using the measurement tool, users could verify that dimension against their CAD file and scale the entire design up or down if it did not match. This practice, which originated from the collective experience of many volunteers at the Invention Studio, was formalized by the researchers into the first heuristic used in this study.

Another important consideration specific to laser cutting is how multiple laser-cut pieces are positioned on a job file relative to each other. It is important not to place multiple cut lines too close to each other, as the region in between is very thin and cannot sufficiently dissipate heat generated by the laser, leading to warping and potential fire risks [24]. This effect is impacted by the kerf of the laser, which is further discussed in Heuristic 6—Kerf.

While the laser-cutting process may be used with a variety of configurations, the most common application utilizes a cutting head mounted to a 2-dimensional Cartesian motion platform, enabling the processing of sheet goods into 2D shapes. In order to produce products that are not constrained to 2D profiles, multiple laser-cut pieces must be assembled in a non-coplanar fashion, and there are a variety of joining processes that may be employed to accomplish this. Typical manufacturing joining methods that are suitable for nonmetallic materials are mechanical fastening and adhesive joining [25]. Of these, adhesive joining can be leveraged to reduce the number of parts required to create a non-coplanar assembly out of 2D parts, but design for manufacturing considerations is required to provide sufficient surface area for adhesion. A collection of examples of laser-cut joinery are found on an Instructables blog post [26], where the user not only considered joinery between rigid materials, such as wood and plastic, but also textiles and other flexible materials. A common joinery method found in laser-cut designs is the finger joint or box joint, in which a series of matching tongues and grooves are cut into the mating edges of multiple parts so that they interlock together like a puzzle piece. Several online tools, such as makercase.com, can generate the files to produce a laser cut box using dimensions, material thickness, and joint parameters input by the designer [27].

Inspirations for laser-cutting design guidelines can be drawn by analogy from other fabrication methods that process sheet goods intended for assembly into complex shapes. Furniture is often manufactured from sheets of wood using a computer numerical control (CNC) router. CNC routing utilizes a cutter, rotated by a spindle, which is maneuvered relative to the workpiece, commonly on a 2-dimensional Cartesian motion platform [28], making it an analogous approach to laser cutting that can be referenced for existing heuristics. Various techniques for joining multiple CNC-machined pieces together are discussed on the furniture design blog OpenDesk [29], as well as an article by Make magazine [30]. These two sources featured multiple examples of joinery methods that could be converted to laser cutter joinery heuristics by analogy, but due to their ease of understanding compared to other methods, finger joints were specifically selected to generate Heuristic 3 about how to join multiple laser-cut pieces together orthogonally.

Another technique used to join multiple laser-cut pieces together is stacking, in which pieces are layered one on top of the other to produce a more complex 3D shape. This procedure was observed primarily in visual model-making. For example, a complex 3D model of a T-Rex head was sliced into a stack of multiple 2D laser-cut pieces and assembled [31]. In this design, it was observed that a matching set of holes was cut into all pieces so that they could be aligned with a dowel when assembling and gluing them together. This practice was formalized into Heuristic 4 regarding how to use alignment features when assembling multiple laser-cut pieces with stacking. The example in this heuristic takes the form of engraving a matching feature in two stacked pieces in order to help line them up during assembly.

When joining multiple laser-cut pieces that fit together tightly, stress can propagate throughout the material and cause failure. An online blog about best practices for laser cutting [24] identified that internal 90-deg corners can become stress concentrators and lead to mechanical failure of parts. It was therefore recommended to insert circular cut-outs into each internal corner. This adds some compliance to the assembly and more evenly distributes stress throughout the bulk material. This recommendation was formalized into Heuristic 5.

Perhaps, the most widely mentioned design consideration for laser cutting is kerf, or the cut width of the laser [32]. The interaction between the laser beam and material can depend on a variety of factors, including the type of lasing medium (CO₂ in this study), material choice, material thickness, laser power, laser speed, and focal distance of the laser lens. The designer must manually measure the kerf by cutting a shape of a known dimension and measuring the difference between the cut dimension and the intended dimension. The designer must apply this difference manually to any design file, shrinking the size of internal features, such as holes, by half the kerf width and expanding the size of external features by half the kerf width [24]. Without kerf compensation, any laser-cut features will differ from the intended dimensions, and this effect is especially noticeable in any joinery or fastening features where the tolerance between multiple laser-cut pieces is important.

When presenting information in an instructional setting, the modality, or method in which information is encoded for presentation to humans, can have a significant effect on the understanding of that material [33]. The modality effect explains that in some cases a “pictoverbal message is better understood when presented in a bimodal (i.e., listening to an illustrated text) rather than unimodal (i.e., reading an illustrated text) way” [34]. There is a varied discussion in the literature regarding the working mechanics of the modality effect, but one common explanation is the Cognitive Load Theory proposed by Sweller et al. [35]; it proposes that by presenting information in a multimodal approach, the cognitive load of understanding that information is distributed across multiple areas in the brain, resulting in a higher level of understanding [35]. The effect of multimodal learning has been studied in the context of general instruction [36,37], where it was found that students learning in a bimodal modality outperformed those learning in a unimodal modality.

Studies have also been done on the effect of heuristic modality on student designs, specifically whether certain modalities of heuristics promote more creativity than others. One study [38] presented a design problem to senior engineering students and varied the modality in which example designs were presented (pictorial vs. text). Results showed that participants who received pictorial examples generated more potential solutions than those that received text; however, participants that received the text examples were more likely to use elements from the examples than those that received pictorial examples [38]. The effect of heuristic modality on designers’ success in producing parts for additive manufacturing has been studied [39] by Fillingim et al. In this study, four design rules for additive manufacturing were presented to both novice and expert designers in four different modalities: Text-Only, text with illustrated example, text with illustrated example, and text with 3D-printed example. This study aimed to understand if varying the modality of rule presentation would affect a designer’s ability to understand those rules. The designers were given these heuristics and asked to complete four-part redesign problems. It was found that while the quality and novelty of participants’ designs did not significantly vary between the four experimental conditions, designers perceived the Text-Only heuristics as the most difficult to understand. While the prior study proves that the modality of rule presentation can influence the understanding of those rules in the context of additive manufacturing, the same has not been studied in the context of laser cutting, which is what this work aims to accomplish.

The participants in this study consisted of undergraduate students in 11 different sections of an introductory mechanical engineering course at the Georgia Institute of Technology. During an 80-min intervention, all participants followed a procedure approved by Georgia Tech’s Institutional Review Board in which they signed a consent form, received an identical lecture on Design for Laser Cutter heuristics, completed a series of surveys, and participated in a redesign activity. The intervention was a part of regular class activities, and there was no additional compensation for participating in the study. If students chose not to sign the consent form, they still participated in the entire intervention, but their data were not included in the results of this study. To protect participant privacy in stored data, each participants’ data was anonymized using an alphanumeric code prior to analysis.

Figure 1 shows a timeline for each group throughout the course of the intervention. The Control group process was conducted in two different class sections, while the rest of the experimental procedures were conducted across three different class sections each. After filling out the consent forms, the participants spent 20 min filling out a Pre-Assessment survey. This survey had three components: a demographic section asking about the participants’ background and relevant experience with laser cutting and design, a self-efficacy section in which participants rated their comfort level with various design and fabrication skills, and a quiz section with questions related to various laser cutter skills.

The self-efficacy survey utilized a 5-point Likert scale with which participants evaluated their level of comfort performing various design-related tasks. The quiz questions were designed to validate whether or not students’ actual abilities increased, along with their self-efficacy, as a result of the intervention. Each quiz answer was assigned a 0 for “Incorrect” or a 1 for “Correct,” and a total quiz score was calculated for each participant by taking the average across all questions.

After completing the Pre-Assessment survey, participants in the three experimental conditions received a lecture on relevant heuristics and design considerations for laser cutting. In addition to the six heuristics previously detailed, the lecture provided an overview of laser cutting, a comparison of laser cutting and traditional manufacturing methods, and discussed the scenarios in which laser cutting is appropriate to use. Each heuristic was then presented on its own slide with a text description, visual aids, and examples. Participants in the Control condition did not receive the lecture at this time in the activity and instead proceeded to the redesign activity, which is further discussed in the Study Material Development section of this paper.

In the redesign activity, the setup varied slightly between the different experimental conditions. Participants in the Control condition received the design prompt and the redesign model, but did not receive any heuristics to use in their redesign. Participants in the Text-Only condition received the design prompt, redesign model, and a sheet of paper with the written heuristics. Participants in the Visual condition received the design prompt, redesign model, and a sheet of paper with the written heuristics along with visual depictions of models where the heuristics were applied. Finally, participants in the Tactile condition received the design prompt, redesign model, and a sheet of paper with the written heuristics along with laser-cut models showing the use case of each heuristic. The full list of heuristics given to participants, along with how they were shown in the Visual and Tactile modalities, may be found in Appendix  A.

All participants, regardless of condition, had 20 min to sketch their redesign. After the sketches were collected by the researcher, all participants completed a Post-Assessment survey. This survey contained the same self-efficacy assessment and quiz questions as the Pre-Assessment, along with an additional section asking participants to rate their comfort level and understanding of the heuristics and provide open feedback to the researchers.

After the Post-Assessment, participants were given 1 week to complete a homework assignment in which they were instructed to create a SOLIDWORKS CAD model of the sketch they generated in class. Students were instructed to take a picture of their sketch to reference while creating the CAD model. The CAD model was used to estimate the total length of the laser cut, which correlates to the amount of fabrication time required.

The redesign problem given to the participants was to redesign a reciprocating mechanism model, as shown in Fig. 2 and described in Appendix  B. The researchers designed and laser cut a simple model of such a mechanism sourced online [40] and purposefully introduced defects during fabrication and assembly that interfered with the operation of the device. Figure 2 shows the resultant design that was given to participants to improve the redesign problem. The participants were given a visual and physical version of this model and tasked to redesign it to improve its functionality, ease of fabrication, and ease of assembly using the laser-cutting heuristics from the presentation.

This specific design was chosen because it is simple enough for novice designers to understand its form and function, but there were many opportunities to improve upon it. For example, because the model must be assembled by hand after fabrication, designers may introduce features into the design to allow for the proper alignment of multiple laser-cut pieces, especially when they need to be stacked on top of each other. In addition, the fact that moving parts are involved means that designers need to carefully consider tolerancing principles and kerf compensation in order to balance the fit tolerance between parts that need to move but not be too loose relative to one another.

In total, 215 students consented to participate in this study; the Text-Only, Visual, and Tactile conditions were made up of three class sections each, with 62, 59, and 57 participants, respectively. The Control condition was made up of two sections, with 37 students. Out of the 215 participants that filled out the demographic survey, 157 identified as men, 56 as women, 1 identified as non-binary, and 1 chose not to identify. Describing the age distribution, 172 participants fell into the 18–20 age group, 38 participants in the 21–23 group, 3 in the 24–26 group, and 1 in the 27–29 group, and 1 in the 30+ group. There were 108 participants who identified as White, 11 as Black, 11 as Hispanic or Latino, 69 as Asian, Native Hawaiian or Other, 1 as American Indian or Alaska Native, 14 as more than one race, and 1 as Other. There were 196 participants pursuing a degree in Mechanical Engineering, 14 pursuing Business majors, 2 pursuing double majors, and 3 pursuing other majors. There was 1 participant in the first year of a degree-seeking program, 94 participants in their second year, 98 in their third year, 19 in their fourth year, and 3 in their fifth year. Lastly, 111 participants identified having no laser-cutting experience prior to this study, and 104 participants identified that they had some prior laser-cutting experience prior to this study, with prior experience being relatively evenly distributed across experimental groups.

The researchers refined a set of metrics based on prior work by Fillingim et al. [39] and Schauer et al. [6,41] to develop five criteria to analyze the quality of the participants’ redesigned parts. For ease of evaluation, the sketches were used for assessment when possible, but some quality metrics could only be evaluated from CAD models. For each of the five criteria, designs were assigned scores of +1, 0, or −1, and each criterion is described as follows.

This category describes the kerf consideration or compensation that is crucial for tolerancing press-fit or tight assemblies. A neutral score of 0 was assigned if the CAD design did not show any interferences, which would mean that the designer did not apply kerf compensation. A score of −1 was assigned if the CAD design showed interferences between parts that were not meant to be a tight fit or press fit, such as the interface between the Slider and Crank. A score of +1 was assigned if the CAD design showed interferences between mating parts that needed to be rigidly assembled, such as the interface between the Handle and Crank.

This category describes the number of laser-cut bodies in the sketched design. A neutral score of 0 was awarded if the redesign kept the number of bodies the same as the original, plus or minus 1. A positive score of +1 was awarded if the redesign reduced the number of bodies by more than 1 compared to the original. This reduction merits a positive score because it translates to an easier fabrication and assembly process, with fewer chances to make mistakes. A negative score of −1 was awarded if the redesign increased the number of bodies by more than 1 compared to the original. This penalty was given because fabricating and assembling more laser-cut pieces means the fabrication process will take more time, and there are more chances of a mistake or error stack-up between the multiple pieces.

This category describes the total length of the laser-cut path and can be correlated to the time it takes to laser cut a certain part. This metric required the use of the participants’ CAD files, since the sketches were not to scale, and the researchers could not make inferences about the actual geometry of the participants’ designs from the sketches alone. In the CAD files, the total perimeter of each laser-cut body could be measured directly and evaluated relative to the base design. If a design’s perimeter fell within 10% of the original in either direction, it was assigned a score of 0. An increase in the perimeter of more than 10% resulted in a negative score of −1, due to the increased fabrication time. A decrease in the perimeter of more than 10% resulted in a positive score of +1.

This category is used to assess any off-the-shelf hardware that participants decided to add to their sketches. The original design was entirely laser cut, but instructions were given to participants during the redesign activity that they could replace parts with off-the-shelf components, such as screws, bolts, nuts, and washers. The reason for this is that oftentimes, laser-cut designs are meant to work in tandem with off-the-shelf hardware for assembling multiple pieces. For this category, a score of −1 was awarded if the design did not use any additional hardware. A score of 0 was awarded if the design used one unique type of hardware. A score of +1 was awarded if the design used more than one unique type of hardware.

This category describes methods of aligning multiple laser-cut pieces to prevent assembly errors. In the original design, there were no alignment features used, resulting in assembly errors that became obvious when the participants tried to use the physical models. First, the wall pieces were misaligned from each other, making the tracks nonparallel and causing the sliding pieces to get stuck when going through a track or when transitioning from one track to another. In addition, the laser-cut pegs used to hold the arm onto the Sliders were sometimes off-center when glued onto the Sliders. Locating features can be used to rectify assembly problems by creating a physical feature on all pieces that need to be assembled. Examples of this include engraving matching marks or cutting matching holes into pieces. For this category, a score of −1 was awarded if the design used no locating features. A score of 0 was awarded if the design used one type of locating feature. A score of +1 was awarded if the design used more than one type of locating feature.

To ensure objectivity in these metrics, two researchers independently rated 25% of the designs for quality. Cohen’s Kappa was used to calculate the inter-rater agreement between the two researchers. For the quality metrics, the researchers achieved 81.7% agreement and a sufficient Cohen’s Kappa of 0.69. Cohen’s Kappa score indicated a robust set of metrics, enabling one researcher to calculate the quality of the remaining designs independently. Scores for each of the five criteria were averaged together and normalized between 0 and 1 to produce an overall quality score for each design.

This category describes the shape of the bottom piece in the design, referred to as the Base. The original design featured a circular base with quarter-circle wall pieces adhered on top. While most participants kept this solution the same in their designs, some opted to switch to a rectangular or square base shape. This change did not impede any functionality of their designs but had the benefit of making the wall pieces easier to align using the corners of the base piece.

This category describes the shape of the sliding pieces in the design, referred to as the Sliders. The original design featured a rectangular Slider with a laser-cut peg glued on top to hold the Crank arm. Most participants kept the Sliders rectangular, but some opted to modify them by adding fillets on the corners or a pointed shape at one end to facilitate easy motion through the tracks. Other participants opted to change the shape of the Sliders to an elliptical or circular shape. These changes reduced friction between the Sliders and walls, and reduced the chance of the assembly getting stuck.

This category describes the method by which the Sliders and the rotating arm, or Crank, were held together. The original design simply uses laser-cut pegs glued on top of the Sliders with corresponding holes in the Crank. This allowed the Crank to freely rotate about each Slider and translate them through their tracks. However, the kerf of the laser made it such that the fit was quite loose. In addition, with nothing to hold the Crank to the Slider, it was easy for the Crank to fall off. Some participants still chose to use laser-cut pegs but switched to a different thickness of acrylic for just these pieces. This meant that the pegs would protrude above the Crank, making it more difficult for it to fall off. Some participants further added to this and placed another laser-cut circle of larger diameter on top of the peg, effectively holding the Crank in place while allowing rotational motion. Other participants replaced the laser-cut pegs with off-the-shelf components, such as screws, nuts, or bolts. These had the benefit of replacing the poorly toleranced laser-cut peg with a standardized part and providing the Crank with a rigid attachment point.

This category was used to take notice of any additional joinery methods that participants used in their designs. Solutions included both laser cutter joinery methods and other methods using off-the-shelf components.

This category was used to further specify where the joinery solutions were applied between two paired pieces. Possible solutions included Base to Walls, Slider to Crank, or Crank to Handle.

Similar to the Quality metrics, two researchers coded 25% of the sketches independently to verify the objectivity and repeatability of the rubric. Overall, the researchers achieved 80.11% agreement, so one researcher coded the remaining data for novelty. A total novelty score was assigned to each sketch by averaging novelty scores for each category, with each category weighted equally.

In this study, participants took a self-efficacy survey and quiz in the Pre-Assessment and Post-Assessment. Based on the literature on the efficacy of design tools in 3D modalities [43], the Tactile group was expected to have the highest increase in self-efficacy between the two surveys since they had a physical piece to reinforce their learning in the heuristics lesson, followed by the Visual group and Text-Only group (Hypothesis 1A). The Control group was expected to have no significant change in self-efficacy since they did not receive the heuristics lesson (Hypothesis 1B). It was expected that the self-efficacy increases would be supported by an increase in performance on the knowledge quiz. Similar to the self-efficacy scores, it was expected that participants in the Tactile group would have the highest increase in quiz scores, followed by the Visual group and Text-Only group (Hypothesis 2A). The Control group was expected to have no significant change in quiz scores since they did not receive the Heuristics lesson (Hypothesis 2B).

With regard to the expected quality of designs, it was expected that the Tactile group would have the highest quality score, followed by the Visual group and the Text-Only group (Hypothesis 3A). The Tactile group was expected to be the most successful because the exposure to the tactile laser-cut models may lead to the translation of more of the heuristics into their designs [39]. The Control group was expected to have the lowest quality score since they had not received a heuristics lesson (Hypothesis 3B).

With respect to design novelty, it has been shown that exposure to tactile models results in a higher degree of design fixation [44]. Therefore, it was believed that participants exposed to the tactile laser-cut models in this study would show the lowest novelty score (Hypothesis 3C), followed by the Visual group and Text-Only group in increasing order. It was also expected that the Control group would have the highest novelty score since they were not given a heuristics lesson and therefore would not experience the design fixation that the other three groups would (Hypothesis 3D).

Depending on the modality of presentation, participants may exhibit preferences for some modalities over others. In the Post-Assessment survey, participants were asked to rate two aspects of each heuristic: how easy the heuristic was to understand conceptually, and how straightforward the heuristic was to apply in the design exercise. It was believed that the Tactile group would have the highest rating in both categories (Hypothesis 4A) due to the presence of a physical model to assist student understanding and application of the heuristics [43]. By the opposite reasoning, the Text-Only group was expected to have the lowest ratings in both categories (Hypothesis 4B).

SPSS Version 26 was used for analyzing the research data and performing statistical analyses. To fully understand how the intervention had an impact on the participants, both within-subjects and between-subjects effects were analyzed using different statistical methods. In within-subjects analysis, data are studied in relation to the same subject. In this study, only data from the Pre-Assessment versus Post-Assessment surveys were analyzed using within-subjects tests. For the within-subjects analysis, the Wilcoxon Signed Rank Test [45] was chosen. This test checks for a statistically significant difference between two data sets measured on the same subject. This is a nonparametric test, which was used because the ordinal data in this study were not normally distributed.

In between-subjects analysis, data are studied with regard to how they vary across independent groups of subjects. In this case, the independent groups are represented by the four experimental groups (Control, Text-Only, Visual, and Tactile), and between-subjects analysis is applied to all dependent variables—the survey data, the quality of designs, and the novelty of designs. For the between-subjects analysis, the Kruskall–Wallis H-test [45] was chosen. This test checks for a statistically significant difference between two or more independently measured groups of subjects relating to a nonparametric dependent variable. The nonparametric test was used because the ordinal data in this study were not normally distributed.

In the Pre-Assessment and Post-Assessment surveys, participants answered self-efficacy questions regarding their level of comfort in performing tasks related to open-ended problem-solving and laser cutting. A 1–5 Likert scale was used, with 1 representing “Extremely Uncomfortable” and 5 representing “Extremely Comfortable.” Out of nine questions, six that closely related to laser-cutting design skills, rather than general design skills, were chosen for aggregate analysis. The Wilcoxon Signed Rank Test revealed that all three experimental groups, as well as the Control group, experienced statistically significant increases (p < 0.001 for all) in their mean self-efficacy ratings from Pre-Assessment to Post-Assessment, as shown in Fig. 3.

In addition to the above analysis, a Kruskall–Wallis H-test was run on the difference between participants’ Pre- and Post-Assessment self-efficacy ratings to check for a statistically significant difference between groups. This test demonstrated that there was a statistically significant difference in the amount by which the groups’ scores increased (χ²(3) = 21.232, p < 0.001), and a pairwise comparison was performed for further detail. This comparison revealed a statistically significant difference between the Control group and all three experimental groups (Control—Tactile, Control—Text-Only, Control—Visual). However, a comparison between the experimental groups (Tactile—Text, Tactile—Visual, Text—Visual) revealed no statistically significant differences.

In the Pre- and Post-Assessment surveys, participants also answered quiz-style questions relating to laser-cutting considerations. The increase in self-efficacy was partially supported by an increase in performance on the quiz. All four experimental groups showed a statistically significant difference between their Pre- and Post-Assessment quiz scores (p < 0.001 for all). While the scores of the Text-Only, Visual, and Tactile groups increased, the Control group quiz scores decreased, as shown in Fig. 4. Again, similarly to the Self-Efficacy results, a Kruskall–Wallis H-test was performed on the difference in Pre- and Post-Assessment quiz scores. The Kruskall–Wallis H-test showed a statistically significant difference in the change in quiz scores across groups (χ²(3) = 65.496, p < 0.001), and a pairwise comparison was performed for further detail. Compared to the Control and Text-Only groups, the Tactile and Visual cases had significantly higher increases in their quiz scores from Pre- to Post-Assessment, with the exception of the comparison between the Text-Only and Tactile groups.

The Kruskal–Wallis H-test on overall quality scores showed no significant differences between groups (χ²(3) = 1.647, p = 0.649), as shown in Fig. 5.

Because the Quality score was composed of multiple categories, a detailed analysis of each category was vital to understand the different design improvements that participants made. Running a Kruskall–Wallis H-test on the quality scores in each category, normalized between 0 and 1, revealed significant differences between groups in two out of five categories: Number of Laser-Cut Bodies (χ²(3) = 8.299, p = 0.040), and Locating Features (χ²(3) = 23.018, p < 0.001), as shown in Fig. 6. The Control group had a significantly higher number of Laser-Cut Bodies quality score than the Tactile group, and all experimental groups had significantly higher Locating Features quality scores than the Control group.

Novelty scores for each category ranged from 0 to 1, based on how many times a particular solution was used by other students. If a particular solution was used by most students, the corresponding Novelty score would trend toward 0, whereas if a particular solution was used by fewer students, the Novelty score would trend toward 1. The Kruskall–Wallis H-test on overall Novelty scores revealed no significant differences between groups (χ²(3) = 6.755, p = 0.080), as shown in Fig. 7.

Similar to Quality, the individual subcategories of the Novelty score were also analyzed separately. The Kruskall–Wallis H-test revealed significant differences in the Slider Geometry (χ²(3) = 10.828, p = 0.013) and Location of Joinery (χ²(3) = 18.726, p < 0.001) categories, as shown in Fig. 8. Pairwise comparisons revealed that the Slider Geometry novelty score for the Tactile group was significantly lower than the Text-Only group. Additionally, the Location of Joinery novelty scores was significantly lower for the Visual group compared to both the Text-Only and Tactile groups.

Participants were asked in the Post-Assessment to rate two aspects of each heuristic on a 1–5 Likert scale: ease of understanding, and ease of application to the redesign activity. The scale was partially anchored, with a rating of 1 corresponding to “Extremely difficult”, 3 was “Neither easy nor difficult,” and 5 was “Extremely easy.” Data for all 6 heuristics were averaged for each of the two questions, and a Kruskall–Wallis H-test was run to determine if any of the three experimental groups had a significantly different perception of the heuristics compared to the others. Both the ease of understanding (χ²(3) = 1.165, p = 0.558) and ease of application (χ²(3) = 3.210, p = 0.201) questions turned out to have no significant difference between experimental groups.

All three experimental groups and the Control group experienced statistically significant increases in their average self-efficacy scores. This supports Hypothesis 1A, with the exception of the Visual group experiencing a higher increase than the Tactile group. It was also not expected that the Control group would also experience a statistically significant increase in the average self-efficacy scores, which contradicts Hypothesis 1B. The participants may have perceived themselves as more skilled in design for laser-cutting techniques from performing the design activity, or they may have been able to “reason through” some heuristics while interacting with the physical part. Their reported increase in self-efficacy may have also been influenced by the good-subject effect, which influences participants to behave in ways that they feel will align with the researchers’ expected outcomes [46]. It should be noted that the Heuristics lesson was also given to the Control group, but after the Post-Assessment, in order to not affect their self-efficacy or quiz scores.

When examining the difference in quiz scores between the Pre-Assessment and Post-Assessment, it was discovered that all three of the experimental groups experienced statistically significant increases in their average quiz scores, partially supporting Hypothesis 2A. Contrary to the hypothesis, the Visual group instead of the predicted Tactile group displayed the greatest quiz score increase. Unexpectedly, the Control group experienced a statistically significant decrease in average quiz scores. This contradicts Hypothesis 2B, which predicted that there would be no significant change in quiz scores in the Control group. Confusion or frustration from being asked to answer the same questions a second time without learning anything in between may have contributed to this decrease in scores. This result shows that although students in the Control group had higher self-efficacy in design for laser-cutting skills, their knowledge of design for laser cutting did not actually increase.

When examining the average quality score across the different groups, no statistically significant differences were found. This contradicts Hypotheses 3A and 3B, which stated that the Tactile group would have the highest quality score, followed by Visual, Text-Only, and Control. One reason for the lack of difference between the scores of the Tactile, Visual, and Text-Only groups could be that participants did not experience a difference in learning with one modality compared to the other two. However, it was unexpected that the Control group had a similar quality score to the other three groups, even though they received no heuristics lesson. The presence of a physical model for the part redesign may have helped the Control group determine what changes needed to be made to the part to increase its quality, or their participation in the Pre-Assessment may have primed them to consider any of their preexisting laser-cutting knowledge.

Since the overall average quality scores did not show any difference, the individual categories that comprised the quality score were further analyzed to see if there were any differences between groups. The first subcategory that showed a significant difference was Number of Laser-Cut Bodies. Compared to the Tactile group, the Control group tended to reduce the number of laser-cut bodies in relation to the original design. This result directly contradicts Hypothesis 3A since, in this case, the Control group had the highest score and the Tactile group the lowest score. One potential factor is that as students in the experimental groups attempted to implement all the heuristics, they made their designs more complex, inherently increasing the number of laser-cut bodies and therefore reducing the quality score associated with this category. There were some designs that managed to implement most of the heuristics without also increasing the number of laser-cut bodies, but the majority were not able to find a solution to this tradeoff. Alternatively, the Control group may have reduced the number of laser-cut bodies in the design in order to avoid designing laser-cut parts due to the lack of knowledge demonstrated in their quiz scores.

Another subcategory that showed a significant difference was Locating Features. In this case, there was a statistically significant difference between the Control group and all three experimental groups, with the Control group having the lowest quality score in this category, and all three experimental groups having higher scores. This result supports Hypothesis 3B, in that the Control group had the lowest score out of the four groups due to not receiving the heuristics lesson. However, while Hypothesis 3A predicted that the highest score would be achieved by the Text-Only group, there was not a significant difference in the scores of the three experimental groups. This is corroborated by the finding that participants did not indicate that any of the heuristic modalities were significantly easier to understand or apply compared to the others.

When examining the average novelty score across the different groups, no statistically significant differences were found. This contradicts Hypothesis 3C and Hypothesis 3D, which predicted that the lowest novelty score would be the Tactile group, followed by the Visual group, Text-Only group, and Control group. Similar to the quality scores, a possible reason for this is that the participants did not exhibit a tendency to understand heuristics better using any one particular modality. Additionally, the part presented for the redesign was relatively simple and may have lacked the opportunity for designers to conduct novel redesigns. However, significant differences were observed in the Slider Geometry, and Location of Joinery novelty subcategories. It was predicted in Hypothesis 3C that the tactile model would promote higher design fixation, which was partially confirmed by the lower Slider Geometry novelty scores of the Tactile group compared to the Text-Only group. For the Location of Joinery, the Visual group had significantly lower novelty scores than the Text-Only group, which supports Hypothesis 3C, as well as the Tactile group, which does not support the hypothesis.

When examining participant ratings for how easy the heuristics were to understand or how easy the heuristics were to apply in the redesign problem, no significant differences were found between the three experimental groups. Although this contradicts Hypothesis 4A and 4B, these results corroborate the findings from the quality score analysis: that the heuristic presentation modality does not impact participants’ ability to apply the heuristics. This result indicates that in engineering education, a visually-presented heuristic may be just as effective as a physically produced heuristic, which could reduce the cost and effort involved in the process of introducing heuristics in educational settings. However, these findings contradict the findings of the study on the presentation modality of design for additive manufacturing heuristics by Fillingim et al. [39], which found that participants perceive text-based heuristics as difficult to understand. It is possible that students’ increased familiarity with subtractive manufacturing processes, such as laser cutting, made the text-based laser-cutting heuristics generally easier to understand compared to the less-familiar additive manufacturing process heuristics.

While this study has made contributions to the field of design and manufacturing education, there were some key limitations that should be discussed. Given the nature of the design problem, it was possible for participants to use multiple solutions to fulfill one novelty category. For example, in the novelty category Location of Joinery, possible answers included Base to Walls, Slider to Crank, and Crank to Handle. Participants could have used one or multiple of these solutions in combination for this one novelty category. The method of quantifying novelty used in this paper was unable to account for the application of multiple solutions. In future work, different methods of analysis should be compared to see if other methods may be more appropriate for this type of data.

An additional aspect of the design problem that could have limited the study was the fact that it was a redesign problem for which students were exposed to a physical model of an existing design that they had to redesign. The scope of potential solutions was limited to the overall architecture of the existing design, reducing the potential novelty of designs. In future studies, a more open-ended design-from-scratch type problem could be explored in the context of design for laser-cutting heuristics to give participants more freedom in generating ideas.

The overlap between the modality groups was also a limitation of this work. While the Text group was shown only text-based heuristics, the Visual and Tactile groups were shown their respective heuristic format in addition to the text-based heuristic. Because two of the heuristics were software-based, they could not be easily represented by a physical part, so the Tactile groups were shown the Visual heuristic in those cases. As a result, the Visual experimental group was using and assessing both the Visual- and Text-based heuristics, while the Tactile group was using and assessing the Tactile, Visual, and Text heuristics, which may have contributed to some of the contradictions between hypotheses and results.

While the size of the study over 200 participants was beneficial for the fidelity of the data analysis, some initial ideas for the study could not be implemented at this scale. For example, an initial idea for this study was to observe participants’ usage of a laser cutter to see how certain heuristics could improve student preparation of laser cutter files and the use of laser cutter software. The time allowance of the in-person intervention did not allow for this idea to be implemented; however, future studies could utilize multiple interventions over a longer period of time to incorporate activities such as preparing designs for laser cutting or adjusting laser-cutting parameters.

It is possible that students could have collaborated during the in-person redesign portion or the CAD modeling homework portion, potentially reducing the overall novelty of the results. This could be caused by the inherent collaborative nature of the class in which the study was conducted. Future work should consider implementing more practices to isolate participants or emphasizing that they work quietly and avoid discussing with each other during the redesign activity.

This study focused solely on design for laser cutting. In real-world problems or designs, multiple fabrication methods such as additive manufacturing, laser cutting, or machining must be used in tandem to produce more complex parts. Future work could broaden the scope of the study to designing parts using multiple fabrication methods and balancing their various strengths and weaknesses to create an effective overall design. This more complex design problem is more reflective of scenarios in that engineering students might find themselves toward the end of their education in a senior design or capstone course, as well as in the professional realm.

This study made key contributions to the formalized presentation of heuristics that assist designers with producing parts suitable for laser cutting. Due to the growing popularity and use of fabrication resources, such as makerspaces in academic settings, it is important to study how best practices in using these resources are presented to novices, such as students. Formalized heuristics have been generated, studied, and researched as training tools for other fabrication methods such as additive manufacturing; however, there was a lack of work done in the scope of laser cutting in academic literature. In this paper, researchers reviewed best practices in other subtractive fabrication methods, as well as online articles and blogs produced by makers, and translated the key information and tips into actionable heuristics.

In addition to the formalization of the design for laser-cutting heuristics, this work also answered the research question posited in the Introduction:

A key contribution of this work was the quantitative analysis of the quality and novelty of designs generated with the assistance of the heuristics in different modalities. These findings contribute to the literature on understanding how heuristics can be beneficial in an educational or training environment. As engineering programs continue to integrate hands-on components into their curricula, it is vital to understand how engineering students and other novice designers can be trained to take advantage of fabrication resources to enrich their education. This study shows that presenting heuristics to novice designers can improve aspects of their designs, as well as their self-efficacy in their ability as a designer, which was supported by a performance improvement on knowledge-based quiz questions.

This work was supported by the U.S. Department of Energy (DOE) award DE-EE0008303. The authors would like to thank the students of ME 2110 and their instructors for allowing them to conduct this study as part of the class, as well as Jaime Berez for his guidance on manufacturing methods.

There are no conflicts of interest.

The data sets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.

Hand sketching is used with notes/annotations to redesign the part in front of you in the next 20 min. The example in front of you has been poorly produced, and you are tasked with adding improvements to it based on the lessons you learned in the heuristics lecture. The image below is also provided as reference, where each colored piece represents a different laser-cut part. If you brainstorm multiple ideas, indicate which is your final design. Notes are used for additional description as necessary and label any key features of the design. You are allowed to add screws, nuts/bolts, washers, adhesives, or dowels, just make a note of them next to your sketch.

Important Specifications:

• Base Circle Diameter: 4″

• Material: 1/8″ thick acrylic