Validation of recombinant growth hormone manufacturing processes

민병조 School of Life Sciences and Biotechnology, Korea U 2010 국내박사

공정밸리데이션이란 “FDA 공정 밸리데이션 가이드라인”에 규정하고 있는 바와 같이 ‘어떤 공정이 미리 설정한 규격과 품질특성 (예를 들면, 역가, 순도, 안정성, 안전성 등)에 적합한 제품을 일관되게 생산할 수 있다는 것을 확실하게 보증할 수 있고 그 결과를 문서화하는 것’을 말한다. 그리고 공정 밸리데이션을 실시하기 전에 제조설비나 기기에 대해서 적격성 확인을 하지 않으면 아니 된다. 인 성장 호르몬을 제조하기 위해서 공정 밸리데이션을 실시하였는데 이는 hGH 발효 및 정제 공정이 각 단계마다 미리 정한 품질에 적합한 원액이 생산되고 또한 재현성을 가지고 있다는 것을 확인하는 것에 그 목적이 있다. 재조합된 인성장호르몬을 생산하기 위하여 교반속도, 온도, 압력, 용존 산소 그리고 pH 조건이 맞춰진 150 L 배양조에서 발효하고, 배양 중 세포의 발현 유도를 위해 Inducer인 3-IAA를 첨가시켰다. 배양이 완료된 액의 세포밀도와 인 성장 호르몬의 발현 유무를 확인하기 위해 O.D 측정 및 SDS-PAGE 전기 영동을 실시하였다. 그 결과 O.D600 80~120에서 세포밀도가 일정하게 유지되어 기준에 적합하였고 표준품과 동일한 위치에 있는 밴드가 확인되었다. 발효가 완료된 액을 세척하고 농축 후 세포 회수율과 농축 볼륨을 측정하였다. 또한 재조합된 세포가 배양조에서 대량생산될 경우 세포내에 불용성의 봉입체가 형성하는데 이를 분리하기 위해서는 세포 파쇄 및 봉입체 분리 과정이 필요하다. 세포 파쇄 후 파쇄율을 측정하고 원심분리 후 침전된 봉입체의 습윤중량을 확인하였다. 그 결과 세포 회수율 및 파쇄율은 90% 이상이고, 습윤중량은 1,500~2,300g으로 기준에 적합하였다. 불용성의 단백질을 활성화시키기 위해서 봉입체를 용해시키기 위하여 교반 시간 및 pH를 조절하여 실시한 결과 pH 10.3~10.7와 교반 시간은 3~5 hr으로 기준 내에 적합하였고, Met-hGH 단백질의 순도 또한 ≥ 50%로 적합함을 확인하였다. 활성화가 완료된 액에 Met-hGH와 다른 불순 단백질과의 분리를 위해 pH를 기준 내에 적합하도록 조절하여 침전이 형성되도록 한 후 수율을 확인한 결과 Met-hGH 단백질 량이 20~45%로 기준에 적합하였다. 활성화 상등액을 1차 음이온교환 크로마토그래피 정제 공정을 실시하고 정제완료액을 채취하여 엔도톡신 (≤ 1000 EU/mg Met-hGH), ECP (≤ 50 ppm), IEF (표준품 주밴드와 이동거리 일치), Native PAGE (Identified, 70% 이상), 순도 (≥ 80%)를 확인한 결과 기준 내에 적합하였고, 수율 또한 20~45%로 기준에 적합하였다. N말단의 met기를 제거하여 천연형 hGH를 생산하기 위해서 전환반응 공정을 실시하였다. 전환반응에서 완충액의 pH는 7.0이며 온도는 44~47℃로 유지되어 기준 내에 적합하게 공정이 진행되었음을 확인하였다. 전환 반응이 완료된 액을 채취하여 엔도톡신 (≤ 800 EU/mg Met-hGH), ECP (≤ 50 ppm), IEF (표준품 주밴드와 이동거리 일치), Native PAGE (Identified, 70% 이상), 순도 (≥ 80%)를 확인한 결과 기준 내에 적합하였고 수율 또한 95~100%로 기준에 적합하였다. N말단의 met기의 제거를 확인하기 위하여 HPLC를 통해 RT (34분) 이후에 피크가 존재하지 않음을 확인하고 전환 반응이 100% 이루어 졌음을 검증하였다. 전환 완료액을 2차 음이온교환 크로마토그래피 정제 공정을 실시하고 정제완료액을 채취하여 엔도톡신 (≤ 20 EU/mg hGH), ECP (≤ 30 ppm), IEF (표준품 주밴드와 이동거리 일치, 주밴드 보다 진해서는 안됨), Native PAGE (Identified, 90% 이상), 순도 (≥ 90 %)를 확인한 결과 기준 내에 적합하였고 수율 또한 60~80%로 기준에 적합하였다. 2차 정제완료액을 역상액체크로마토그래피 정제 공정을 실시하고 정제완료액을 채취하여 IEF (표준품 주밴드와 이동거리 일치, 주밴드 보다 진해서는 안됨), Native PAGE (Identified, 92% 이상), 순도 (≥ 92%)를 확인한 결과 기준 내에 적합하였고 수율 또한 80~95%로 기준에 적합하였다. 투석을 실시하고 완료된 액을 3차 음이온교환 크로마토그래피 정제 공정을 실시하고 정제완료액을 채취하여 엔도톡신 (≤ 5 EU/mg hGH), ECP (≤ 10 ppm), IEF (표준품 주밴드와 이동거리 일치, 주밴드 보다 진해서는 안됨), Native PAGE (Identified, 98% 이상), 순도 (≥ 94%)를 확인한 결과 기준 내에 적합하였고 수율 또한 80~95%로 기준에 적합하였다. 마지막으로 Manufacturing Bulk 공정에서는 정제가 완료된 원액이 정해진 허용기준에 적합하게 최종의 제형화 공정을 거쳐 완료되었음을 나타내었다. 공정 밸리데이션 결과 본 생산공정은 재현성 있게 정해진 품질에 적합한 제조 공정이라 할 수 있겠다. 본 연구는 인 성장 호르몬의 산업화된 생산 공정을 밸리데이션하므로서 단백질 의약품의 산업화와 최적생산공정을 개발하는데 도움을 줄 것이며 더 나아가 본 연구는 단백질 의약품의 대량생산체계를 검증하는데 좋은 예가 될 수 있다 Validation of Recombinant Human Growth Hormone (r-hGH) Manufacturing Processes by Byeong Jo Min Process validation defined by “Guidance of Process Validation in FDA” is to assure absolutely that some process consistently produce the suitable product at the already fixed standard and specification (titer, purity, stability and safety etc), and to document with these results. Also, it must be confirmed the eligibility test on the facilities and equipments for the production of interest before process validation. It has been taken process validation for r-hGH manufacturing, and its purpose is to confirm that the fermentation and purification processes for the production of the r-hGH solutions that fulfill the pre-established criteria at each step, and get the reproducibility. For the production of r-hGH, cultivation was carried out in 150 L fermentor operated in the fitted conditions such as agitation speed, temperature, pressure, dissolved oxygen, pH, and 3-IAA as inducer for leading cell expression during For the production of r-hGH, cultivation was carried out in 150 L fermentor operated in the fitted conditions such as agitation speed, temperature, pressure, dissolved oxygen, pH, and 3-IAA as inducer for leading cell expression during cultivation was added into the culture broth. After fermentation, when the cell density of culture broth and SDS-PAGE to confirm the r-hGH expression were measured, O.D600 was kept in standard values between 80 and 120, and the band obtained from SDS-PAGE was detected at the same position comparing with standard band. The cell recovery ratio and concentration volume were measured after the cells produced were washed and concentrated by using micro-filtration and then, the cell disruption ratio and the humidity weight of precipitate were also measured after the cell was disrupted and centrifuged to isolate the insoluble inclusion body. The results obtained from on each step were satisfied with the acceptance criteria pre-established as follows; cell recovery ratio: over 90%, cell disruption ratio: over 90%, humidity weight of precipitate: 1,500~2,300 g. Inclusion body was dissolved to activate the insoluble protein by controlling stirring time and pH. After the dissolution of inclusion body, stirring time (3~5 hr), pH (10.3~10.7) and the purity of Met-hGH protein (over 50%) fit in the standard. When the activated r-hGH solution was treated with phosphoric acid and precipitated and centrifuged to separate between Met-hGH protein and impurity protein, the yield Met-hGH (20~45%) of supernatant was also satisfied with the acceptance criteria pre-established. The activated supernatant was loaded into the first anion exchange chromatography and eluted with buffer. When the eluate was assessed in accordance with the pre-established sampling plan and quality control tests, the results satisfied the acceptance criteria pre-established for the following purification process as follows; endotoxin: ≤ 350 EU/mg, ECP: ≤ 42 ppm, IEF: same removal distance, r-hGH content in Native-PAGE: > 73%, purity by HPLC: ≥ 86%, purification yield by UV scanning: 32~36%. Therefore, the first anion exchange chromatography process was well validated and consistently yields crude r-hGH solutions that fulfill the pre-established criteria for the next purification process. For producing natural hGH, the methyl group of N-terminal was removed in conversion process with immobilized enzyme. Buffer pH and temperature in conversion reaction were 7.0, 44~47℃, respectively, in standard value. When the conversion reaction was completed, endotoxin (≤ 800 EU/mg Met-hGH), ECP (≤ 50 ppm), IEF (fitness in main band), Native PAGE (identified, over 70%) and purity (≥80%) tests were exactly in standard value and the yield was also in 95~100%. Also, HPLC was performed to identify the removal of methyl group of N-terminal. We checked that there was no signal after RT 34 min and also confirmed that conversion reaction was completed perfectly. The converted solution was used in the second anion exchange chromatography. The results of endotoxin (≤ 20 EU/mg hGH), ECP (≤ 30 ppm), IEF (fitness in main band, not to be strong than main band), Native PAGE (Identified, over 90%) and purity (≥ 90%) were in approval standard, and the yield was in 60~80%. Secondary completed solution was taken to Reverse-Phase chromatography and then took some sample. The results of IEF (fitness in main band, not to be stronger than main band), Native PAGE (Identified, over 92%) and purity (≥ 92%) were in standard, and the yield was in 80~95%. I took it to go via the dialysis. After test, it was taken to the third anion exchange chromatography and then took some sample. The results of endotoxin (≤ 5 EU/mg hGH), ECP (≤ 10 ppm), IEF (fitness in main band, not to be strong than main band), Native PAGE (Identified, over 98%) and purity (≥ 94%) were in approval standard, and the yield was in 80~95%. Finally in manufacturing bulk process, raw solution which the last purification was completed showed that it was in approval standard and was absolutely taken to final product process. Therefore, the consequence of process validation is that this process has reproducibility and is approval for high quality product as protocol. I think that this study would help being the industrialization of protein medical supplies and developing the best product process as r-hGH taken to industrial validation. Furthermore, this study could be good instance to confirm bulk production of protein medical supplies.

Validation of HSGC method for the determination of residual solvents in active pharmaceutical ingredients

김영석 Graduate School, Korea University 2012 국내석사

Residual solvents in pharmaceuticals are organic volatile impurities left over from the synthesis of active pharmaceutical ingredients (APIs) or from the manufacturing processing of pharmaceutical products. Their levels are monitored and controlled for safety reasons and for their potential impacts on the crystalline form, possibly affecting solubility, stability, and bioavailability. Therefore, the analytical method for the control of the residual solvent should be developed and validated. Residual solvents originated from the production process of APIs (Erdosteine and Ticlopidine hydrochloride) were methanol, ethanol, acetone, isopropanol (IPA), ethyl acetate (EA) and chloroform. Headspace gas chromatographic (HSGC) method was developed and validated for the quantification of residual solvents in APIs. To analyze the residual solvent of APIs, APIs must be equally dissolved in a certain solvent. However, depending on the types, some APIs are water soluble while others are water insoluble. Therefore, to test APIs with different solubility in residual solvent analysis, it is required to perform dimethylformamide (DMF) which has high boiling point, good solubilizing, high purity and thermal stability. In order to extract residual solvent more effectively and to improve precision, a mix of water and DMF is used to dissolve APIs. The optimum condition for headspace sampler were: incubation time = 60 min, oven temperature = 80℃, and equilibrium time = 20min. After the analytical method was fully developed, it was validated. The method validation was done by evaluating specificity, accuracy, precision, linearity, limit of detection (LOD), limit of quantitation(LOQ) and range as indicated in the International Conference on Harmonization (ICH) guideline, Q2 Validation of Analytical Procedures: Text and Methodology. The validation studies showed that newly developed the analytical method was specific for the residual solvents, methanol, ethanol, acetone, IPA, ethyl acetate and chloroform. The analytical method was proved linear in the range. The correlation coefficients (R2) were more than 0.99. The LOQs of methanol, ethanol, acetone, IPA, EA and chloroform were 47.4 ppm, 58.1 ppm, 7.9 ppm, 31.2 ppm, 20.6 ppm and 8.4 ppm respectively. The average recovery rate (accuracy) of ethanol, acetone, IPA and chloroform for Erdosteine were 101.3 %, 101.6%, 99.1% and 99.3% respectively. The average recovery rate (accuracy) of methanol and EA for Ticlopidine hydrochloride were 100.5 % and 100.6 % respectively. The repeatability precision of methanol was 6.88 % RSD and intermediate precision was 5.73 % RSD. The repeatability precision of ethanol was 6.64 % RSD and intermediate precision was 6.78 % RSD. The repeatability precision of acetone was 8.45 % RSD and intermediate precision was 2.64 % RSD. The repeatability precision of IPA was 7.67 % RSD and intermediate precision was 5.12 % RSD. The repeatability precision of EA was 5.61 % RSD and intermediate precision was 3.71 % RSD. The repeatability precision of chloroform was 9.83 % RSD and intermediate precision was 9.22 % RSD. These precision values were below the USP limit of 15 % RSD. This study will help to develop test method of residual solvent in API with water solubility and insolubility. Moreover, this study will be a good example of method validation for the residual solvent analysis of APIs in compliance with global standards.

Probabilistic validation and computer -aided debugging in analog/mixed- signal systems : pre-silicon global convergence property checking and post-silicon bug localization

윤상호 서울대학교 대학원 2014 국내박사

Increasing system complexity, growing uncertainty in semiconductor technology, and demanding requirements in complex specifications pose significant challenges to both pre-silicon design verification and post-silicon chip validation. Thus, this dissertation investigates efficient pre-silicon/post-silicon validation and debugging methodology, especially for analog and mixed-signal (AMS) systems. Principally, validation is formulated as a Bayesian inference problem and analyzed in a probabilistic manner. For instance, pass/fail property can be checked by Bayesian sampling – the posterior distribution of the unknown failure probability can be measured after many sample validation trials so as to quantify the confidence of pass with a given tolerance and model accuracy. This approach is first taken in the pre-silicon verification to check a system’s property. In other words, the efficient Monte Carlo-based methods for ensuring global convergence property are proposed using two techniques: fast sample batch verification using cluster analysis and efficient sampling using Gaussian process regression. In addition, a practical design flow for preventing global convergence failure is presented – the notion of indeterminate state X is extended to AMS systems. For the post-silicon validation, in particular, the probabilistic graphical model is proposed as one effective abstraction of AMS systems. Using the probabilistic graphical model and statistical inference, we can compute the probability of each parameter to satisfy a given specification and use it for bug localization and ranking. The proposed model and method are especially useful at the post-silicon validation phase, since they can check and localize bugs in the system under limited observability and controllability.

Propiverine hydrochloride의 품질기준 적합성을 위한 분석방법의 Validation

최미희 강원대학교 산업대학원 2011 국내석사

생리식염키트주사 제조에 대한 Process Validation

최충원 아주대학교 산업대학원 2008 국내석사

제약 시장의 Validation이란 "공정, 시설 또는 시스템이 의도한 대로 적절히 기능하고 있는지를 확인하기 위하여 이들을 체계적으로 조사·검토하여 문서화하는 것" 이며, "Validation 되었다" 라는 것은 품질규격에 적합한 제품이 일관되게 제조된다는 것이 증명되어 정식으로 승인된 것이며, 어느 특정한 공정, 방법, 기계설비 또는 시스템이 미리 설정되어 있는 판정기준에 적합한 결과를 얻는다는 것을 검증하고 이를 문서화하는 것을 말한다. Process Validation 지침으로는 첫째, 품질, 안정성 및 유효성은 제품 중에 설계되어야 한다. 둘째, 품질은 최종제품의 시험만으로는 보증할 수 없다. 셋째, 제조공정의 각 단계는 최종 제품이 모든 품질 및 설계 규격에 합치되도록 괸리 되어야 한다. 본 연구의 목적은 생리식염키트주사의 Process Line 따른 적격성평가의 보증여부에 대하여 Process Validation 을 실시 하여 판정 방법을 제시하고자 하였다. 우선 판정 절차를 체계적이고 과학적으로 수행하기 위하여, 검증기관으로부터 보증된 실험기기와 생산기기를 사용하였다. 조제 · 여과 · 세척 · 충전 · 밀봉 · 원형고리융착 · 커버씰융착 · 멸균 · 라벨 · 포장 공정 별로 Protocol 을 작성하여 각 공정의 적격성 평가를 문서화된 SOP 및 대한약전을 통하여 시험 및 실험을 실시하였다. Process Parameter 의 범위 설정의 근거를 마련하기 위해 조제 공정과 자재 세척과정에서 Range Study 를 실시하였으며, 이에 따라 공정 별 결과를 확인 및 검토하여, Parameter 의 최적화 조건을 설정하였다. 3 Lot 의 Process Validation 을 실시한 후 품질에 영향을 끼칠 수 있는 Worst Case Study 를 실시하여 그에 따른 제품의 영향을 줄 수 있는 요지를 찾아 실험하였다. 이에 따라, 모든 공정 변수(Parameter) 가 제조 및 공정 동안 설정기준에 적합하였으며, 최적화되어 설계된 품질 특성과 규격에 맞는 제품을 일관되게 생산함을 증명하여 유효성과 안정성이 확보된 우수한 의약품 제조를 입증 할 수 있는 과학적 자료가 되리라 기대한다. Meaning of “Validation” at pharmacy market is to examine progress, facility or system function as intended way and make documents of the result. “It is done validation” means it is certificated by quality standard and is made into documents. Guides for Process validation are the following. Firstly, quality, stability and availability have to be made at process of design. Secondly, quality is not insured by only examination of finished products. Thirdly, each phase of process has to be managed appropriate to standards of quality and design. The Purpose of this study is showing definitive ways by process validation at the saline solution kit injection process line. First of all, to practice definition process systematically, the used experiment utensils and production utensils were insured by certification institute. Compounding of medicine-filtration-washing-filling-sealing-attaching round shaped ring-attaching covered seal-sterilization-Labeling After making protocol which is appropriate to each packing process, the study examined eligibility of each process through documented SOP and Korea pharmacopoeia. Range study at processes of preparing medicines and washing was done to get fundamental range of process parameter. Accordingly to the analysis of the result, parameter’s optimization conditions were set. After process validation of three batched, worst case study also conducted, which can affects product quality. Therefore, all processes’ parameters are appropriate to standard at making process and they are optimized. So, I expect these results can be scientifically applied to define medicines production processes that insures stability and availability.

A Framework for Validating Decompilers and Profiling Execution Inefficiency of Ethereum Smart Contracts

Jeong, Seongho 연세대학교 일반대학원 2026 국내박사

검증서버 그룹핑 기법을 이용한 실시간 인증서 상태 검증 프로토콜

최선묵 東國大學校 2005 국내박사

PKI(Public Key Infrastructure) allowing use of Public Key Cryptograph and certificates is required to meet the security requirements in the open network and distributed network environment. A key task in PKI is Certificate Status Validation. Certificates can be revoked even before the validity period expires, for reasons such as changes in the certificate holder's personal data, damages to or exposure of the private key and cancellation of user authority, and they can be revoked on the expiration of the validity period. Certificate Status Validation is an important and mandatory task for the validating party based on the PKI. Certificate validation methods under the PKI environment include the CRL(Certificate Revocation List), OCSP(On-line Certificate Status Protocol), SCVP(Simple Certificate Validation Protocol) and DVCS(Data Validation and Certification Server). CRL contains problems in that revocation information is created on a regular basis, real-time referencing is not supported, and the traffic is increased along with file sizes. Although the OCSP Server method provides real-time validation, validation process efficiency could be compromised with the increasing number of users. Distributed OCSP server method can cause problems including distribution CRL, load on a particular server, slow validation process, and consistency and security concerns. This paper suggests ways to resolve some of the problems raised in the existing methods, using distributed OCSP server based on group, such as reducing load, maintaining consistency and receiving and transmitting data with enhanced security. This paper proposed a model for conducting certificate validation procedures, which had been processed by a single OCSP server, by grouping a number of distributed OCSP servers. Experiments indicated that the Distributed OCSP Server based on Group method outperforms other methods in terms of traffic reduction and average service request response time. The main reason for the proposed model not yielding a better response time is the duration required for receiving certificate revocation information issued by the CA(Certification Authority) in real time. In the proposed model, consistency is the key. All OCSP servers must have the same information at all times and CA allows all OCSP servers to use the updated information only after it receives the confirmation message from all OCSP servers. There are a few important factors in the proposed model. First is the reduction of traffic. Traffic of all OCSP servers in the same group will be measured based on the threshold and the server with the least load will perform the validation service, thereby reducing the traffic. Second is consistency. CA transmits the updated Updated CRL to all OCSP servers in the group and conducts the validation service only after it has received the message that the information has been received successfully by all OCSP servers. Third is security. When CA and OCSP servers send and receive information on certificate suspension and revocation, CA uses private key for encryption and transmission, and the recipient OCSP server uses public key to decryptand validate the information, thereby ensuring the security. Lastly, in the all or nothing method used to maintain consistency, a time gap can be created when receiving the confirmation message, depending on the status of individual OCSP servers. Due to this time gap, an OCSP server in normal operation can be disregarded or the transmitted Updated CRL can be ignored, which is an issue that needs to be further discussed.

Optimization-based Model Improvement for Error Sources Identification in a Computational Model

손혜정 서울대학교 대학원 2021 국내박사

The increased use of computer-aided engineering (CAE) in recent years requires a more accurate prediction capability in computational models. Therefore, extensive studies have considered engineering strategies to achieve highly credible computational models. Optimization-based model improvement (OBMI), which includes model calibration, validation, and refinement, is one crucial technique that has emerged to enhance the prediction ability of computational models. Model calibration is the process of estimating unknown input parameters in a computational model. Model validation presents a judgement of the accuracy of a predicted response. If it is possible for a computational model to have model form uncertainties, model refinement explores unrecognized error sources of a computational model. OBMI can adopt these three processes individually or sequentially, according to the trustworthiness of the prior knowledge of the computational modeling. Although OBMI process improvements have emerged to try to consider the major sources of errors, OBMI can still suffer from a failure to improve a computational model. Since numerous error sources in an experimental and computational model are intertwined with each other, OBMI has difficulty identifying the error sources required to enable accurate prediction ability of the computational model. Thus, eventually, OBMI may fail to propose an appropriate solution. To cope with this challenge, this doctoral dissertation research addresses three essential issues: 1) Research Thrust 1 – a new experimental design approach for model calibration to reduce parameter estimation errors; 2) Research Thrust 2) – a device bias quantification method for considering model form errors with bound information; and, Research Thrust 3) – comparison of statistical validation metrics to consider type II errors in model validation. Research Thrust 1: A variety of sources of errors in observation and prediction can interrupt the model improvement process. These error sources degrade the parameter estimation accuracy of the model calibration. When a computational model turns out to be invalid because of these error sources, the OBMC process performs model refinement. However, since model validation cannot distinguish between parameter estimation errors and modeling errors, it is difficult for the existing method to efficiently refine the computational model. Thus, this study aims to develop a model improvement process that identifies the leading cause of invalidity of a prediction. In this work, an experimental design method is integrated with optimization-based model improvement to minimize the effect of estimation errors in model calibration. Through use of the proposed method, after calibration, the computational model mainly includes the effects of unrecognized modeling errors. Research Thrust 2: The experimental design method proposed in Research Thrust 1 has the advantage of being able to identify two error sources without additional observation. However, model calibration still suffers from parameter estimation errors, since experimental design is affected by model form errors. The parameters estimated by model calibration are often unreasonable for engineers in practical settings because they have expert-based prior knowledge about the model parameters. Among the variety of physical information available, bound information about model parameters is a suitable constraint in optimization-based model calibration (OBMC). Using prior information about parameter bounds, Research Thrust 2 devises proportionate bias calibration to quantify the amount of degradation of the predicted responses that is due to model form errors in a computational model. The bias term is estimated in the optimization-based model calibration (OBMC) algorithm with unknown parameters to enable OBMC to support accurate estimation of unknown parameters within a prior bound. This study proposes a new formulation of a bias term that depends on the output responses to resolve the gap in appropriate bias that arises due to the different dimensions of the predicted responses. Research Thrust 3: Statistical model validation (SMV) evaluates the accuracy of a computational model’s predictions. In SMV, hypothesis testing is used to determine the validity or invalidity of a prediction, based on the value of a statistical validation metric that quantifies the difference between the predicted and observed results. Errors in hypothesis testing decisions are troublesome when evaluating the accuracy of a computational model, since an invalid model might be used in practical engineering design activities and incorrect results in these settings may lead to safety issues. This research compares various statistical validation metrics to highlight those that show fewer errors in hypothesis testing. The resulting work provides a statistical validation metric that is sensitive to a discrepancy in the mean or variance of the two distributions from the predictions and observations. Statistical validation metrics examined in this study include Kullback-Leibler divergence, area metric with U-pooling, Bayes factor, likelihood, probability of separation, and the probability residual. 컴퓨터 이용 공학 기술의 활용도가 증가함에 따라, 각 공학분야에서는 보다 정확한 예측 능력을 가진 컴퓨터 모델을 필요로 하게 되었다. 많은 연구결과를 통해, 신뢰도 높은 계산모델을 얻기 위한 공학기술들이 개발되었다. 최적화 기반 모델 향상 기술은 계산모델 예측도 향상을 위한 공학기술 중 하나로, 모델 보정, 모델 검증, 그리고 모델 개선 과정을 포함하고 있다. 모델 보정은 계산 모델 내 미지변수의 값을 역으로 추정하는 기술이다. 모델 검증은 예측 성능의 정확도를 판단한다. 계산모델 내 미지 오류 원인이 존재하면 모델 개선을 통해 미지 원인을 탐색하는 작업을 수행한다. 최적화 기반 모델향상기술 내 세가지 세부 기술들은 모델 관련 사전 정보의 양에 따라 유기적으로, 혹은 개별적으로도 수행이 가능하다. 모델 향상 기술이 계산모델 내 영향을 주는 다양한 오류원인을 고려하여 수행되고 있으나, 최적화 기반 모델 향상기술은 여전히 계산모델의 정확도를 증가시키는데 한계점을 지니고 있다. 시험 데이터 및 계산 모델 내 다양한 오류 소스들이 결합되어 있어, 최적화 기반 모델 향상 기술은 이 오류원인들을 구분하고 각 오류원인들에 대해 적합한 솔루션을 제공하기에 부적합하다. 따라서, 이러한 문제점을 해결하고자 본 박사학위논문에서는 (1) 파라미터 추정 오류 감소를 위한 시험 설계 기법, (2) 모델 보정 시 모델링 및 시험 오류의 양을 정량화 하기 위한 비율 편향도 정량화 기법 (3) 2종 오류에 강건한 통계기반 검증 척도 비교 연구를 제안하고자 한다. 첫 번째 연구에서는 파라미터 추정 오류를 최소화하기 위한 시험 설계법 개발을 목표로 한다. 여기서 결정된 시험설계안은 모델 보정 시 사용될 시험 데이터 취득을 위한 시험 설계를 뜻한다. 계산모델 내 발생하는 모델링 오류, 그리고 시험데이터 취득 시 발생하는 계측오류 등은 모델 보정에서 정확한 파라미터 값의 추정을 방해한다. 파라미터 추정 오류를 포함한 계산모델은 주어진 시험데이터를 잘 모사하는 것처럼 보이지만, 파라미터를 과도하게 편향된 값으로 추정하여 모델링 오류를 보완한 결과이다. 이 경우, 모델 검증 시 모델이 유효하다고 판단될 수 있지만 실제로는 파라미터 추정오류와 모델링 오류를 동시에 갖고 있으므로 다양한 설계조건에서 유효하지 않은 모델이다. 따라서, 본 연구에서는 파라미터 추정오류와 모델링 오류를 구분하여 정확한 모델 검증을 유도하고자 한다. 파라미터 추정오류와 모델링 오류는 그 정도를 각각 정량화 하는 것이 불가능하므로, 파라미터 추정오류를 가장 최소화 할 수 있는 시험데이터의 종류와 취득위치를 선정할 수 있는 시험설계법을 고안하였다. 이를 위해, (1) 파라미터 추정오류를 수식적으로 유도하였고, (2) 유도된 식 내에서 사용자가 제어할 수 있는 일부항을 최소화 하도록 하였다. 제안된 시험설계법은 파라미터 추정오류와 모델링 오류를 구분하고, 모델 검증 시 유효 및 불유효의 원인이 모델링오류가 될 수 있도록 한다. 두 번째 연구에서는 파라미터 추정오류를 개선하기 위해 모델 보정 시 모델링 오류에 의한 성능 저하량을 정량화 할 수 있는 비율 편향 보정 기법을 제안한다. 첫 번째 연구에서 제안한 시험설계법은 별도의 추가 시험 데이터 없이 파라미터 추정 오류와 모델링 오류를 구분해 낼 수 있는 최선의 방법론 이지만, 모델링 오류 및 시험 오류의 영향이 큰 경우 파라미터 추정오류를 획기적으로 개선하는데 한계가 있다. 오류의 영향도가 큰 모델은 추정 파라미터의 값이 엔지니어가 가진 경험, 혹은 물리 기반 정보에 위배되는 지점으로 수렴할 수 있다. 따라서, 본 연구에서는 관측데이터 외 미지 모델 변수의 물리적 정보를 활용하여 모델링 오류 및 관측오류에 의한 성능저하도의 양을 정량화 하고자 한다. 연구에서 제안된 ‘비율편향’ 은 오류에 의한 성능저하도를 성능값의 일정한 비율로 가정하여, 모델 보정 시 최적화 알고리즘 내에서 미지모델변수와 함께 최적 값이 추정되는 항이다. 비율편향 항과 미지모델 변수가 사전의 물리적 정보에 위배되지 않는 범위 내에서 추정될 수 있도록 미지모델 변수의 범위 정보를 최적화 알고리즘의 제한조건으로 활용한다. 비율편향 보정기법은 미지모델변수의 추정값이 모델링 오류에 의한 성능저하를 보완하기 위해 과도하게 편향된 값으로 최적화 되는 현상을 바로잡을 수 있다. 세 번째 연구에서는 모델 검증 시 발생할 수 있는 결정 오류를 개선하기 위해 통계적 검증 척도의 선택 기준을 제시하고자 한다. 모델 검증은 주로 통계기반 방법인 가설검증을 활용하여 모델의 유효 및 불유효를 결정한다. 가설검증은 제 1종 오류 및 제 2종 오류의 발생 가능성을 갖고 있다. 제 2종 오류는 불유효한 모델을 유효하다고 판단하는 오류로써 실제 산업분야에 치명적인 사고를 유발할 수 있다. 본 연구에서는 제 2종 오류를 가장 적게 발생 시킬 수 있는 통계적 검증 척도를 분석하기 위해 다음과 같은 조건에서의 검증 정확도 비교 연구를 수행한다. 1) 관측 및 예측 성능의 분산이 같고 평균값의 차이로 인해 예측 성능이 불유효 한 경우, 2) 관측 및 예측성능의 평균보다 분산값의 차이로 인해 예측 성능이 불유효 한 경우. 비교연구는 모델 파라미터의 분산 정도를 4가지로 세분화 하고 관측 데이터 개수에 의한 정확도 차이를 비교하고자 관측 데이터를 3개에서 30개까지 증가시켰다. 그 결과, 성능 간 평균의 차이를 잘 정량화 하는 검증척도 및 성능 간 분산의 차이를 잘 정량화 하는 검증척도를 제안할 수 있었다. 제안된 검증척도의 평균지향 및 분산지향 특성을 증명하고자, 평균지향 척도의 극한값을 유도하여 분산값의 증가 시 척도의 값이 최대값에 도달하지 않아 검증 오류가 발생할 수 있음을 확인하였다.

Verified Translation Validation For Register Promotion in LLVM

박상훈 서울대학교 대학원 2018 국내석사

Mainstream C/C++ compilers usually focus on efficiency though reliability is also important as well for compilers. Thus they perform various optimizations for performance, but many bugs are found in GCC and LLVM by recent random testing tools. To increase the reliability of mainstream compilers without sacrificing their performance, I propose a practical approach for verified compilation using a verified extensible relational Hoare-logic (ERHL) validator. As this approach performs translation validation which separates compilation and validation, it does not sacrifice any compilation performance. Also, I can finally succeed to validate translations or identify compiler bugs because I insert proof generation code for validation directly in the compiler. As a litmus test to show the possibility of this approach, I successfully validate register promotion in the sroa pass in LLVM 3.7.1. Translation validation for register promotion has enough to show possibility of validating the whole optimization passes in LLVM as register promotion is one of the most performance critical passes in LLVM. This is because register promotion promotes memory locations to LLVM registers and converts corresponding load and store instructions into register read and write operations, and many LLVM optimization focus on optimizing register operations, most of which are introduced in register promotion. Moreover, some of those LLVM registers are transformed into CPU registers which are magnitude faster to access than memory locations. I validated translation of register promotion for SPEC CPU2006 benchmarks, LLVM Nightly Test, and five C projects, with 7.1M lines of code in total. There are only 10 failures in validation results, but all of them are due to the compiler bug which I found during this work.

Population-based modeling for predicting preterm birth and its clinical applications : A decade of national health insurance data in Korea

차재우 Graduate School, Korea University 2026 국내박사