Java tutorial
/* * * * * Apache License * Version 2.0, January 2004 * http://www.apache.org/licenses/ * * TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION * * 1. Definitions. * * "License" shall mean the terms and conditions for use, reproduction, * and distribution as defined by Sections 1 through 9 of this document. * * "Licensor" shall mean the copyright owner or entity authorized by * the copyright owner that is granting the License. * * "Legal Entity" shall mean the union of the acting entity and all * other entities that control, are controlled by, or are under common * control with that entity. For the purposes of this definition, * "control" means (i) the power, direct or indirect, to cause the * direction or management of such entity, whether by contract or * otherwise, or (ii) ownership of fifty percent (50%) or more of the * outstanding shares, or (iii) beneficial ownership of such entity. * * "You" (or "Your") shall mean an individual or Legal Entity * exercising permissions granted by this License. * * "Source" form shall mean the preferred form for making modifications, * including but not limited to software source code, documentation * source, and configuration files. * * "Object" form shall mean any form resulting from mechanical * transformation or translation of a Source form, including but * not limited to compiled object code, generated documentation, * and conversions to other media types. * * "Work" shall mean the work of authorship, whether in Source or * Object form, made available under the License, as indicated by a * copyright notice that is included in or attached to the work * (an example is provided in the Appendix below). * * "Derivative Works" shall mean any work, whether in Source or Object * form, that is based on (or derived from) the Work and for which the * editorial revisions, annotations, elaborations, or other modifications * represent, as a whole, an original work of authorship. For the purposes * of this License, Derivative Works shall not include works that remain * separable from, or merely link (or bind by name) to the interfaces of, * the Work and Derivative Works thereof. * * "Contribution" shall mean any work of authorship, including * the original version of the Work and any modifications or additions * to that Work or Derivative Works thereof, that is intentionally * submitted to Licensor for inclusion in the Work by the copyright owner * or by an individual or Legal Entity authorized to submit on behalf of * the copyright owner. For the purposes of this definition, "submitted" * means any form of electronic, verbal, or written communication sent * to the Licensor or its representatives, including but not limited to * communication on electronic mailing lists, source code control systems, * and issue tracking systems that are managed by, or on behalf of, the * Licensor for the purpose of discussing and improving the Work, but * excluding communication that is conspicuously marked or otherwise * designated in writing by the copyright owner as "Not a Contribution." * * "Contributor" shall mean Licensor and any individual or Legal Entity * on behalf of whom a Contribution has been received by Licensor and * subsequently incorporated within the Work. * * 2. Grant of Copyright License. Subject to the terms and conditions of * this License, each Contributor hereby grants to You a perpetual, * worldwide, non-exclusive, no-charge, royalty-free, irrevocable * copyright license to reproduce, prepare Derivative Works of, * publicly display, publicly perform, sublicense, and distribute the * Work and such Derivative Works in Source or Object form. * * 3. Grant of Patent License. Subject to the terms and conditions of * this License, each Contributor hereby grants to You a perpetual, * worldwide, non-exclusive, no-charge, royalty-free, irrevocable * (except as stated in this section) patent license to make, have made, * use, offer to sell, sell, import, and otherwise transfer the Work, * where such license applies only to those patent claims licensable * by such Contributor that are necessarily infringed by their * Contribution(s) alone or by combination of their Contribution(s) * with the Work to which such Contribution(s) was submitted. If You * institute patent litigation against any entity (including a * cross-claim or counterclaim in a lawsuit) alleging that the Work * or a Contribution incorporated within the Work constitutes direct * or contributory patent infringement, then any patent licenses * granted to You under this License for that Work shall terminate * as of the date such litigation is filed. * * 4. Redistribution. You may reproduce and distribute copies of the * Work or Derivative Works thereof in any medium, with or without * modifications, and in Source or Object form, provided that You * meet the following conditions: * * (a) You must give any other recipients of the Work or * Derivative Works a copy of this License; and * * (b) You must cause any modified files to carry prominent notices * stating that You changed the files; and * * (c) You must retain, in the Source form of any Derivative Works * that You distribute, all copyright, patent, trademark, and * attribution notices from the Source form of the Work, * excluding those notices that do not pertain to any part of * the Derivative Works; and * * (d) If the Work includes a "NOTICE" text file as part of its * distribution, then any Derivative Works that You distribute must * include a readable copy of the attribution notices contained * within such NOTICE file, excluding those notices that do not * pertain to any part of the Derivative Works, in at least one * of the following places: within a NOTICE text file distributed * as part of the Derivative Works; within the Source form or * documentation, if provided along with the Derivative Works; or, * within a display generated by the Derivative Works, if and * wherever such third-party notices normally appear. The contents * of the NOTICE file are for informational purposes only and * do not modify the License. You may add Your own attribution * notices within Derivative Works that You distribute, alongside * or as an addendum to the NOTICE text from the Work, provided * that such additional attribution notices cannot be construed * as modifying the License. * * You may add Your own copyright statement to Your modifications and * may provide additional or different license terms and conditions * for use, reproduction, or distribution of Your modifications, or * for any such Derivative Works as a whole, provided Your use, * reproduction, and distribution of the Work otherwise complies with * the conditions stated in this License. * * 5. Submission of Contributions. Unless You explicitly state otherwise, * any Contribution intentionally submitted for inclusion in the Work * by You to the Licensor shall be under the terms and conditions of * this License, without any additional terms or conditions. * Notwithstanding the above, nothing herein shall supersede or modify * the terms of any separate license agreement you may have executed * with Licensor regarding such Contributions. * * 6. Trademarks. This License does not grant permission to use the trade * names, trademarks, service marks, or product names of the Licensor, * except as required for reasonable and customary use in describing the * origin of the Work and reproducing the content of the NOTICE file. * * 7. Disclaimer of Warranty. Unless required by applicable law or * agreed to in writing, Licensor provides the Work (and each * Contributor provides its Contributions) on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or * implied, including, without limitation, any warranties or conditions * of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A * PARTICULAR PURPOSE. You are solely responsible for determining the * appropriateness of using or redistributing the Work and assume any * risks associated with Your exercise of permissions under this License. * * 8. Limitation of Liability. In no event and under no legal theory, * whether in tort (including negligence), contract, or otherwise, * unless required by applicable law (such as deliberate and grossly * negligent acts) or agreed to in writing, shall any Contributor be * liable to You for damages, including any direct, indirect, special, * incidental, or consequential damages of any character arising as a * result of this License or out of the use or inability to use the * Work (including but not limited to damages for loss of goodwill, * work stoppage, computer failure or malfunction, or any and all * other commercial damages or losses), even if such Contributor * has been advised of the possibility of such damages. * * 9. Accepting Warranty or Additional Liability. While redistributing * the Work or Derivative Works thereof, You may choose to offer, * and charge a fee for, acceptance of support, warranty, indemnity, * or other liability obligations and/or rights consistent with this * License. However, in accepting such obligations, You may act only * on Your own behalf and on Your sole responsibility, not on behalf * of any other Contributor, and only if You agree to indemnify, * defend, and hold each Contributor harmless for any liability * incurred by, or claims asserted against, such Contributor by reason * of your accepting any such warranty or additional liability. * * END OF TERMS AND CONDITIONS * * APPENDIX: How to apply the Apache License to your work. * * To apply the Apache License to your work, attach the following * boilerplate notice, with the fields enclosed by brackets "[]" * replaced with your own identifying information. (Don't include * the brackets!) The text should be enclosed in the appropriate * comment syntax for the file format. We also recommend that a * file or class name and description of purpose be included on the * same "printed page" as the copyright notice for easier * identification within third-party archives. * * Copyright 2016 Alibaba Group * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. * * */ package com.taobao.android.builder.tools.proguard; import java.io.File; import java.io.IOException; import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map; import com.alibaba.fastjson.JSON; import com.android.build.gradle.internal.api.AppVariantContext; import com.android.build.gradle.internal.api.AwbTransform; import com.google.common.collect.Lists; import com.taobao.android.builder.AtlasBuildContext; import com.taobao.android.builder.dependency.model.AwbBundle; import com.taobao.android.builder.tasks.app.BuildAtlasEnvTask; import com.taobao.android.builder.tools.FileNameUtils; import com.taobao.android.builder.tools.ReflectUtils; import com.taobao.android.builder.tools.cache.FileCacheCenter; import com.taobao.android.builder.tools.cache.FileCacheException; import com.taobao.android.builder.tools.log.FileLogger; import com.taobao.android.builder.tools.proguard.domain.Input; import com.taobao.android.builder.tools.proguard.domain.Result; import com.taobao.android.builder.tools.proguard.dump.BundleProguardDumper; import com.taobao.android.builder.tools.proguard.dump.ClazzRefInfo; import com.taobao.android.builder.tools.zip.ZipUtils; import org.apache.commons.io.FileUtils; import org.apache.commons.lang3.StringUtils; import org.jetbrains.annotations.NotNull; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import proguard.Configuration; import proguard.ConfigurationParser; import proguard.ParseException; import proguard.ProGuard; import proguard.classfile.ClassPool; import static com.taobao.android.builder.tools.proguard.AwbProguardConfiguration.INJARS_OPTION; import static com.taobao.android.builder.tools.proguard.AwbProguardConfiguration.OUTJARS_OPTION; import static proguard.AtlasProguardConstants.INOUT_CFG; /** * Created by wuzhong on 2017/5/13. */ public class BundleProguarder { public static final String CACHE_TYPE = "proguard-bundles-0.12"; private static Logger logger = LoggerFactory.getLogger(BundleProguarder.class); private static FileLogger fileLogger = FileLogger.getInstance("proguard"); public static void execute(AppVariantContext appVariantContext, Input input) throws Exception { if (input.proguardOutputDir != null && input.proguardOutputDir.exists()) { FileUtils.cleanDirectory(input.proguardOutputDir); } if (!appVariantContext.getAtlasExtension().getTBuildConfig().isProguardCacheEnabled()) { doProguard(appVariantContext, input); return; } String md5 = input.getMd5(); Result result = loadProguardFromCache(appVariantContext, input); String bundleName = input.getAwbBundles().get(0).getAwbBundle().getName(); if (result.success) { fileLogger.log(bundleName + " hit cache " + result.cacheDir.getAbsolutePath()); return; } fileLogger.log(bundleName + " miss cache " + result.cacheDir.getAbsolutePath()); long startTime = System.currentTimeMillis(); doProguard(appVariantContext, input); double during = (System.currentTimeMillis() - startTime) / 1000.0; fileLogger.log(bundleName + "proguard consume (s) " + during); //cache if (null != result.cacheDir) { try { cacheProguard(appVariantContext, input, result); } catch (Throwable e) { logger.error(e.getMessage(), e); } } } public static Result loadProguardFromCache(AppVariantContext appVariantContext, Input input) throws Exception { Result result = new Result(); String md5 = input.getMd5(); result.key = input.getAwbBundles().get(0).getAwbBundle().getName() + "_" + md5; logger.info("bundle proguard for " + result.key + " with md5 key " + md5); File cacheDir = FileCacheCenter.queryFile(CACHE_TYPE, result.key, true, isRemoteCacheEnabled(appVariantContext)); result.cacheDir = cacheDir; if (null == cacheDir || !cacheDir.exists()) { logger.warn("bundle proguard for " + result.key + " miss cache " + cacheDir.getAbsolutePath()); return result; } logger.warn("bundle proguard for " + result.key + " hit cache "); File keepJson = new File(cacheDir, "keep.json"); File proguardCfg = new File(cacheDir, "proguard.cfg"); File usageCfg = new File(cacheDir, "usage.cfg"); Map<String, File> md5Map = input.getMd5Files(); if (input.getAwbBundles().get(0).getAwbBundle().isMainBundle()) { AtlasBuildContext.atlasMainDexHelperMap.get(appVariantContext.getVariantName()).getMainDexFiles() .clear(); AtlasBuildContext.atlasMainDexHelperMap.get(appVariantContext.getVariantName()).getInputDirs().clear(); for (File file : cacheDir.listFiles()) { if (file.getName().endsWith("jar") && ZipUtils.isZipFile(file)) { String jarMd5 = file.getName().replace(".jar", ""); File srcFile = md5Map.get(jarMd5); if (null != srcFile && srcFile.exists()) { String fileName = FileNameUtils.getUniqueJarName(srcFile); FileUtils.copyFile(file, new File(input.proguardOutputDir, fileName + ".jar")); AtlasBuildContext.atlasMainDexHelperMap.get(appVariantContext.getVariantName()) .addMainDex(new BuildAtlasEnvTask.FileIdentity(fileName, new File(input.proguardOutputDir, fileName + ".jar"), false, false)); } else { FileUtils.copyFileToDirectory(file, input.proguardOutputDir); AtlasBuildContext.atlasMainDexHelperMap.get(appVariantContext.getVariantName()) .addMainDex(new BuildAtlasEnvTask.FileIdentity(file.getName(), new File(input.proguardOutputDir, file.getName()), false, false)); } } } File awbProguardDir = input.printConfiguration.getParentFile(); //FileUtils.copyFileToDirectory(keepJson, awbProguardDir); if (proguardCfg.exists()) { FileUtils.copyFileToDirectory(proguardCfg, awbProguardDir); } if (usageCfg.exists()) { FileUtils.copyFileToDirectory(usageCfg, awbProguardDir); } result.success = true; return result; } else { if (!keepJson.exists()) { logger.error("bundle proguard for " + result.key + " missing keep.json "); FileUtils.deleteDirectory(cacheDir); return result; } Map<AwbTransform, List<File>> transformListMap = new HashMap<>(); for (AwbTransform awbTransform : input.getAwbBundles()) { File awbProguardDir = appVariantContext.getAwbProguardDir(awbTransform.getAwbBundle()); FileUtils.copyFileToDirectory(keepJson, awbProguardDir); if (proguardCfg.exists()) { FileUtils.copyFileToDirectory(proguardCfg, awbProguardDir); } if (usageCfg.exists()) { FileUtils.copyFileToDirectory(usageCfg, awbProguardDir); } List<File> inputFiles = new ArrayList<>(); transformListMap.put(awbTransform, inputFiles); List<File> files = new ArrayList<>(); files.addAll(awbTransform.getInputLibraries()); //configs.add(); if (null != awbTransform.getInputDir() && awbTransform.getInputDir().exists()) { files.add(awbTransform.getInputDir()); } for (File oldFile : files) { File file = new File(cacheDir, input.getFileMd5s().get(oldFile) + ".jar"); if (file.exists()) { if (ZipUtils.isZipFile(file)) { inputFiles.add(file); } } else { logger.error("miss proguard jar for :" + file.getAbsolutePath()); FileUtils.deleteDirectory(cacheDir); logger.error("delete cache Dir " + cacheDir.getAbsolutePath()); return result; } } } for (AwbTransform awbTransform : input.getAwbBundles()) { awbTransform.setInputFiles(transformListMap.get(awbTransform)); awbTransform.setInputDir(null); awbTransform.getInputLibraries().clear(); awbTransform.getAwbBundle().setKeepProguardFile(keepJson); } result.success = true; return result; } } private static boolean isRemoteCacheEnabled(AppVariantContext appVariantContext) { return appVariantContext.getAtlasExtension().getTBuildConfig().isProguardNetworkCacheEnabled(); } public static void doProguard(AppVariantContext appVariantContext, Input input) throws Exception { Configuration configuration = parseConfiguration(appVariantContext, input); configuration.dump = input.dump; if (null == input.printConfiguration) { configuration.printConfiguration = new File( appVariantContext.getAwbProguardDir(input.getAwbBundles().get(0).getAwbBundle()), "proguard.cfg"); } else { configuration.printConfiguration = input.printConfiguration; } configuration.printSeeds = null; configuration.dump = null; configuration.obfuscate = false; // configuration.optimizationPasses = 1; configuration.printUsage = input.printUsage; configuration.printMapping = input.printMapping; // Execute ProGuard with these options. ProGuard proGuard = new ProGuard(configuration); System.out.println(">>> start to do proguard"); proGuard.execute(); System.out.println("<<< end proguard"); ClassPool classPool = (ClassPool) ReflectUtils.getField(proGuard, "programClassPool"); Map<String, ClazzRefInfo> clazzRefInfoMap = BundleProguardDumper.dump(proGuard, input.getDefaultLibraryClasses()); //Fileoutputs for (AwbTransform awbTransform : input.getAwbBundles()) { if (!awbTransform.getAwbBundle().isMainBundle()) { AwbBundle awbBundle = awbTransform.getAwbBundle(); File fileOut = new File(appVariantContext.getAwbProguardDir(awbBundle), "keep.json"); fileOut.delete(); fileOut.getParentFile().mkdirs(); FileUtils.write(fileOut, JSON.toJSONString(clazzRefInfoMap)); awbBundle.setKeepProguardFile(fileOut); } } } @NotNull private static Configuration parseConfiguration(AppVariantContext appVariantContext, Input input) throws IOException, ParseException { Configuration configuration = new Configuration(); List<File> proguardFiles = new ArrayList<>(); proguardFiles.add(printInOut(appVariantContext, input)); proguardFiles.addAll(input.getDefaultProguardFiles()); proguardFiles.addAll(input.getParentKeeps()); for (File file : proguardFiles) { ConfigurationParser parser = new ConfigurationParser(file, System.getProperties()); try { parser.parse(configuration); } finally { parser.close(); } } for (File file : input.getLibraryProguardFiles()) { KeepOnlyConfigurationParser parser = new KeepOnlyConfigurationParser(file, System.getProperties()); try { parser.parse(configuration); } finally { parser.close(); } } return configuration; } private static File printInOut(AppVariantContext appVariantContext, Input input) throws IOException { File proguardDir = input.proguardOutputDir; if (null == proguardDir) { proguardDir = appVariantContext.getAwbProguardDir(input.getAwbBundles().get(0).getAwbBundle()); } proguardDir.mkdirs(); File inoutConfigs = new File(proguardDir, INOUT_CFG); List<String> configs = new ArrayList<>(); for (AwbTransform awbTransform : input.getAwbBundles()) { List<File> inputLibraries = Lists.newArrayList(); for (File inputLibrary : awbTransform.getInputLibraries()) { configs.add(INJARS_OPTION + " " + inputLibrary.getAbsolutePath()); String name = FileNameUtils.getUniqueJarName(inputLibrary); File obsJar = new File(proguardDir, name + ".jar"); input.proguardJarMap.put(obsJar, input.getFileMd5s().get(inputLibrary)); inputLibraries.add(obsJar); configs.add(OUTJARS_OPTION + " " + obsJar.getAbsolutePath()); if (inputLibrary.isDirectory() && AtlasBuildContext.atlasMainDexHelperMap .get(appVariantContext.getVariantName()).getInputDirs().contains(inputLibrary)) { input.maindexFolderTransform.put(inputLibrary, obsJar); } else if (AtlasBuildContext.atlasMainDexHelperMap.get(appVariantContext.getVariantName()) .inMainDex(inputLibrary)) { input.maindexFileTransform.put(inputLibrary, obsJar); // AtlasBuildContext.atlasMainDexHelper.updateMainDexFile(inputLibrary, obsJar); } else if (awbTransform.getAwbBundle().isMainBundle()) { AtlasBuildContext.atlasMainDexHelperMap.get(appVariantContext.getVariantName()) .addMainDex(new BuildAtlasEnvTask.FileIdentity(obsJar.getName(), obsJar, false, false)); } } //configs.add(); if (null != awbTransform.getInputDir() && awbTransform.getInputDir().exists()) { configs.add(INJARS_OPTION + " " + awbTransform.getInputDir().getAbsolutePath()); String name = input.getFileMd5s().get(awbTransform.getInputDir()); if (null == name) { name = FileNameUtils.getUniqueFileName(awbTransform.getInputDir().getName(), "proguard"); } File obsJar = new File(proguardDir, name + ".jar"); inputLibraries.add(obsJar); configs.add(OUTJARS_OPTION + " " + obsJar.getAbsolutePath()); if (awbTransform.getAwbBundle().isMainBundle()) { AtlasBuildContext.atlasMainDexHelperMap.get(appVariantContext.getVariantName()).getInputDirs() .clear(); AtlasBuildContext.atlasMainDexHelperMap.get(appVariantContext.getVariantName()) .addMainDex(new BuildAtlasEnvTask.FileIdentity(obsJar.getName(), obsJar, false, false)); } } awbTransform.setInputFiles(inputLibraries); awbTransform.setInputDir(null); awbTransform.getInputLibraries().clear(); } for (File library : input.getLibraries()) { configs.add("-libraryjars " + library.getAbsolutePath()); } FileUtils.writeLines(inoutConfigs, configs); return inoutConfigs; } private static void cacheProguard(AppVariantContext appVariantContext, Input input, Result result) throws FileCacheException, IOException { AwbBundle awbBundle = input.getAwbBundles().get(0).getAwbBundle(); File tmpDir = new File(appVariantContext.getScope().getGlobalScope().getIntermediatesDir(), "progurad-cache/" + awbBundle.getName()); tmpDir.mkdirs(); for (AwbTransform awbTransform : input.getAwbBundles()) { for (File file : awbTransform.getInputFiles()) { if (file.exists()) { String md5 = input.proguardJarMap.get(file); if (StringUtils.isNoneEmpty(md5)) { FileUtils.copyFile(file, new File(tmpDir, md5 + ".jar")); } else { FileUtils.copyFileToDirectory(file, tmpDir); } } else { new File(tmpDir, file.getName()).createNewFile(); } } } //Keep cache. Json File keepFile = awbBundle.getKeepProguardFile(); if (null != keepFile && keepFile.exists()) { FileUtils.copyFileToDirectory(keepFile, tmpDir); } File proguardCfg = new File(null != input.proguardOutputDir ? input.proguardOutputDir : appVariantContext.getAwbProguardDir(awbBundle), "proguard.cfg"); if (!proguardCfg.exists()) { proguardCfg = input.printConfiguration; } if (null != proguardCfg && proguardCfg.exists()) { FileUtils.copyFileToDirectory(proguardCfg, tmpDir); } File usageCfg = new File(null != input.proguardOutputDir ? input.proguardOutputDir : appVariantContext.getAwbProguardDir(awbBundle), "usage.cfg"); if (!usageCfg.exists()) { usageCfg = input.printUsage; } if (null != usageCfg && usageCfg.exists()) { FileUtils.copyFileToDirectory(usageCfg, tmpDir); } FileCacheCenter.cacheFile(CACHE_TYPE, result.key, tmpDir, AtlasBuildContext.sBuilderAdapter.pushCacheToNetwork && isRemoteCacheEnabled(appVariantContext)); } }