2000 2000 k = 256 1800 1800 k = 128 1600 k = 64 1400 Objective function value Objective function value 1600 1200 k=8 1000 800 1400 1200 1000 800 600 600 400 400 200 200 0 0 5 10 Number of Iterations 15 0 0 50 100 150 Number of clusters 200 250 300 k = 512 5000 5000 4500 k = 256 4500 4000 Objective function value Objective function value 4000 k = 64 3500 3000 2500 k=8 2000 3500 3000 2500 2000 1500 1500 1000 1000 500 500 0 0 5 10 15 Number of Iterations 20 25 30 0 0 100 200 300 Number of clusters 400 500 600 8 k=8 7 6 5 k=64 4 k=512 3 2 1 0 0 0.1 20 k=512 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.7 0.8 0.9 1 18 k=64 16 14 k=8 12 10 8 6 4 2 0 0 0.1 0.2 0.3 0.4 0.5 0.6 16 10 14 9 12 7 Singular Values Singular Values 8 6 5 4 3 10 8 6 4 2 2 1 0 0 50 100 150 Index 200 250 0 0 50 100 150 Index 200 4 7000 2.4 x 10 2.2 6000 2 Approximation Error Approximation Error 5000 Clustering 4000 3000 Best(SVD) 1.8 Clustering 1.6 1.4 1.2 2000 Best(SVD) 1 1000 0 50 100 150 Number of vectors 200 250 0.8 0 50 100 150 200 250 300 Number of vectors 350 400 450 500 4 4000 1.3 x 10 Random 1.25 Random 3500 Approximation Error Approximation Error 1.2 3000 Concept Decompositions 2500 1.15 1.1 Concept Decompositions 1.05 1 Best(SVD) Best(SVD) 0.95 2000 0.9 1500 0 50 100 150 Number of vectors 200 250 0.85 0 50 100 150 200 250 300 Number of vectors 350 400 450 500 0.5 0.4 0.3 0.2 0.1 0 −0.1 −0.2 0 500 1000 1500 2000 2500 3000 3500 4000 500 1000 1500 2000 2500 3000 3500 4000 500 1000 1500 2000 2500 3000 3500 4000 0.5 0.4 0.3 0.2 0.1 0 −0.1 −0.2 0 0.5 0.4 0.3 0.2 0.1 0 −0.1 −0.2 0 0.5 0.4 0.3 0.2 0.1 0 −0.1 −0.2 0 500 1000 1500 2000 2500 3000 3500 4000 500 1000 1500 2000 2500 3000 3500 4000 500 1000 1500 2000 2500 3000 3500 4000 0.5 0.4 0.3 0.2 0.1 0 −0.1 −0.2 0 0.5 0.4 0.3 0.2 0.1 0 −0.1 −0.2 0 1 0.9 0.9 0.8 0.8 Fraction of Nonzeros in Concept Vectors Fraction of Nonzeros in Concept Vectors 1 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.7 0.6 0.5 0.4 0.3 0.2 0.1 50 100 150 Number of clusters 200 250 300 0 0 100 200 300 Number of clusters 400 500 600 1 0.9 0.9 Average inner product between Concept Vectors Average inner product between Concept Vectors 1 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 50 100 150 Number of clusters 200 250 300 0 0 100 200 300 Number of clusters 400 500 600 1 1 0.9 0.9 k=8 k = 16 k = 256 0.8 0.8 0.7 0.7 Cosine of Principal Angles Average Cosine of Principal Angles k = 32 0.6 0.5 0.4 0.3 0.6 0.5 k = 64 0.4 0.3 0.2 0.2 0.1 0.1 0 0 50 100 150 Number of Clusters 200 0 0 250 1 10 0.9 0.9 k = 16 0.8 0.8 0.7 0.7 0.6 0.5 0.4 0.3 30 40 Number of Principal Angles 50 k = 64 k = 128 0.5 0.4 0.3 0.2 0.1 0.1 50 100 150 Number of Clusters 200 k = 256 k = 32 0 0 250 1 1 0.9 0.9 0.8 0.8 0.7 0.7 50 100 150 Number of Principal Angles 200 k=8 250 k = 256 k = 16 k = 128 Cosine of Principal Angles Average Cosine of Principal Angles 60 0.6 0.2 0 0 20 1 Cosine of Principal Angles Average Cosine of Principal Angles k = 128 0.6 0.5 0.4 0.3 0.5 k = 64 0.4 0.3 0.2 0.2 0.1 0.1 0 0 50 100 150 Number of Singular Vectors 200 250 k = 32 0.6 0 0 10 20 30 40 Number of Principal Angles 50 60 1 0.9 0.9 0.8 0.8 0.7 0.7 0.6 0.5 0.4 0.3 k = 256 k = 64 0.4 0.3 0.2 0.1 0.1 50 100 150 200 250 300 Number of Clusters 350 400 450 k = 128 0.5 0 0 500 1 10 20 30 40 Number of Principal Angles 50 60 1 0.9 k = 16 0.8 0.8 0.7 0.7 Cosine of Principal Angles 0.9 0.6 0.5 0.4 0.3 k = 32 k = 512 k = 64 0.6 0.5 0.4 k = 256 0.3 k = 128 0.2 0.2 0.1 0.1 0 0 50 100 150 200 250 300 Number of Clusters 350 400 450 k = 235 0 0 500 1 1 0.9 0.9 0.8 0.8 0.7 0.7 Cosine of Principal Angles Average Cosine of Principal Angles k = 16 0.6 0.2 0 0 Average Cosine of Principal Angles k = 512 k=8 k = 32 Cosine of Principal Angles Average Cosine of Principal Angles 1 0.6 0.5 0.4 0.3 k = 235 k = 16 k = 32 k = 128 0.4 0.3 0.1 200 k=8 k = 64 0.1 100 150 Number of Singular Vectors 200 0.5 0.2 50 100 150 Number of Principal Angles 0.6 0.2 0 0 50 0 0 10 20 30 40 Number of Principal Angles 50 60 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0 0.3 0.25 0.2 0.15 0.1 0.05 0 −0.05 −0.1 −0.15 −0.2 0